← BACK TO MAIN PATHPARALLEL HORIZONS
LESSON 03MULTI-GPU SCALING
READ
  1. 01CAPACITY
  2. 02SHARD
  3. 03COLLECT
  4. 04COMPLETE

STEP 01 · CAPACITY

THE KERNEL HASN'T RUN. MEMORY IS ALREADY FULL.

The new dataset and model state total 24 GB, while one GPU in this teaching system has 16 GB of memory. Try the allocation and locate the bottleneck.
SINGLE-GPU PROGRAM1 DEVICE · 24 GB WORKLOAD
01cudaSetDevice(0);
02float* state;
03cudaMalloc(&state, 24_GB);
04train<<<grid, block>>>(state);
ALLOCATE→KERNEL

24 GB / 16 GB is a teaching scenario that expresses the capacity relationship, not a specification for every P100 configuration.