How does GPT math actually work end to end? Follow one training step from token IDs through Q/K/V, multi-head attention, MLP, logits, cross-entropy loss, backprop, and AdamW — past O = Attention(Q,K,V).
One full transformer training step worked by hand — embeddings, positional encoding, attention, layer norm, the encoder-decoder, cross-entropy loss, backprop, and Adam.
A verification-first analysis of circulating DeepSeek V4 claims, plus the real mathematics behind sparse attention, KV-cache scaling, stable residual mixing, and long-context training.
How does attention work in transformers? A visual, step-by-step explanation of the attention mechanism — queries, keys, values, the attention pattern, masking, the low-rank value trick, and multi-head attention — with the real GPT-3 parameter counts.