DeepSeek V4 Inside: One Token Through Every Block
How does DeepSeek V4 process one token? I follow the byte p in deepseek through CSA, HCA, mHC, MoE, and MTP on a toy width, with Flash-0731 sizes.
Looped Transformers Explained: Recurrent Depth and Astra
What is a looped transformer? How recurrent depth reuses layers, why Nanbeige 4.2 runs a 22-layer stack twice, and why that does not hide chain of thought.
Build a Mini LLM from Scratch in NumPy: RoPE, GQA, SwiGLU
How to build a mini LLM from scratch in NumPy: RoPE, GQA, QK-Norm, SwiGLU, tied embeddings, and a 3.87M chat companion with full architecture visuals.
DeepSeek V4 Explained: Long-Context Engineering and Math
A verification-first look at DeepSeek V4 claims, plus the real math behind sparse attention, KV-cache scaling, and long-context training stability.
How Attention Works in Transformers: Queries, Keys, Values
How does attention work in transformers? Queries, keys, values, the attention pattern, masking, and multi-head attention, with real GPT-3 parameter counts.
How GPT Actually Works: A Visual Guide to Transformers
How does a GPT actually work? A visual walkthrough of transformers: tokens, word embeddings, dot products, softmax and temperature, with real GPT-3 numbers.
Build GPT-2 from Scratch in PyTorch: A Full Walkthrough
How to build GPT-2 from scratch in PyTorch: tokenization, causal self-attention, transformer blocks, weight tying, and a 124M training loop that runs.