Standard attention is slow because it writes an N×N score matrix to GPU HBM. FlashAttention never materializes that matrix: it tiles Q, K, and V into SRAM, uses online softmax, and still produces the exact same output — which is why long-context training and inference became practical.