← BACK TO MAIN PATHPARALLEL HORIZONS
LESSON 04TENSOR COMPUTE
READ
  1. 01PROFILE
  2. 02TILE
  3. 03PRECISION
  4. 04COMPLETE

STEP 01 · PROFILE

THERE ARE MANY THREADS. MATRIX MULTIPLICATION STILL REPEATS THE SAME SMALL OPERATION.

Each Thread owns one output element, yet must perform 16 multiply-adds in sequence along K. The program is correct, but it does not express matrix structure to the hardware.
SCALAR MATRIX MULTIPLY256 OUTPUT THREADS · K = 16
01int row = blockIdx.y * blockDim.y + threadIdx.y;
02int col = blockIdx.x * blockDim.x + threadIdx.x;
03float sum = 0.0f;
04for (int k = 0; k < 16; k++) {
05sum += A[row][k] * B[k][col];
06}
07C[row][col] = sum;
SCALAR KERNEL READY0 / 256
OUTPUT TILE C16 × 16 ELEMENTS
FMA PER OUTPUT16OUTPUT THREADS256MATRIX SHAPE16³

The animation shows computational structure, not hardware clocks, throughput, or a fixed speedup.