DeepSeek V4 Inside: One Token Through Every Block
How does DeepSeek V4 process one token? I follow the byte p in deepseek through CSA, HCA, mHC, MoE, and MTP on a toy width, with Flash-0731 sizes.
Looped Transformers Explained: Recurrent Depth and Astra
What is a looped transformer? How recurrent depth reuses layers, why Nanbeige 4.2 runs a 22-layer stack twice, and why that does not hide chain of thought.
Natural Language Inference Explained: Entailment in NLP
What is natural language inference (NLI)? How models label a premise and hypothesis as entailment, contradiction, or neutral on SNLI, MNLI, and BERT.
Build a Mini LLM from Scratch in NumPy: RoPE, GQA, SwiGLU
How to build a mini LLM from scratch in NumPy: RoPE, GQA, QK-Norm, SwiGLU, tied embeddings, and a 3.87M chat companion with full architecture visuals.
GPT Math Explained: The Full Forward Pass Beyond Attention
How does GPT math work end to end? One training step from token IDs through Q/K/V, attention, MLP, logits, cross-entropy loss, backprop, and AdamW.
Positional Encoding Explained: How Transformers Learn Order
What is positional encoding in transformers? Why attention is order-blind, learned vs sinusoidal embeddings, and a from-scratch NumPy implementation.
Transformer from Scratch: Forward Pass and Backprop by Hand
One full transformer training step worked by hand: embeddings, positional encoding, attention, layer norm, cross-entropy loss, backprop, and Adam.
How Attention Works in Transformers: Queries, Keys, Values
How does attention work in transformers? Queries, keys, values, the attention pattern, masking, and multi-head attention, with real GPT-3 parameter counts.
How GPT Actually Works: A Visual Guide to Transformers
How does a GPT actually work? A visual walkthrough of transformers: tokens, word embeddings, dot products, softmax and temperature, with real GPT-3 numbers.
Build GPT-2 from Scratch in PyTorch: A Full Walkthrough
How to build GPT-2 from scratch in PyTorch: tokenization, causal self-attention, transformer blocks, weight tying, and a 124M training loop that runs.