DeepSeek V4 Inside: One Token Through Every Block
How does DeepSeek V4 process one token? I follow the byte p in deepseek through CSA, HCA, mHC, MoE, and MTP on a toy width, with Flash-0731 sizes.
Looped Transformers Explained: Recurrent Depth and Astra
What is a looped transformer? How recurrent depth reuses layers, why Nanbeige 4.2 runs a 22-layer stack twice, and why that does not hide chain of thought.
GPT Math Explained: The Full Forward Pass Beyond Attention
How does GPT math work end to end? One training step from token IDs through Q/K/V, attention, MLP, logits, cross-entropy loss, backprop, and AdamW.
Transformer from Scratch: Forward Pass and Backprop by Hand
One full transformer training step worked by hand: embeddings, positional encoding, attention, layer norm, cross-entropy loss, backprop, and Adam.
DeepSeek V4 Explained: Long-Context Engineering and Math
A verification-first look at DeepSeek V4 claims, plus the real math behind sparse attention, KV-cache scaling, and long-context training stability.
How Attention Works in Transformers: Queries, Keys, Values
How does attention work in transformers? Queries, keys, values, the attention pattern, masking, and multi-head attention, with real GPT-3 parameter counts.