Build a Mini LLM from Scratch in NumPy: RoPE, GQA, SwiGLU Visual Guide
How to build a mini LLM from scratch in NumPy — RoPE, GQA, QK-Norm, SwiGLU, tied embeddings, and a 3.87M chat companion with full architecture visuals.
DeepSeek V4 Claims Examined: Long-Context Engineering, Math, and Verification
A verification-first analysis of circulating DeepSeek V4 claims, plus the real mathematics behind sparse attention, KV-cache scaling, stable residual mixing, and long-context training.
How Attention Works in Transformers: Queries, Keys and Values
How does attention work in transformers? A visual, step-by-step explanation of the attention mechanism — queries, keys, values, the attention pattern, masking, the low-rank value trick, and multi-head attention — with the real GPT-3 parameter counts.
How GPT Actually Works: A Visual Guide to Transformers
How does a GPT actually work? A plain-English, visual walkthrough of transformers — tokens, word embeddings, why directions in vector space mean something, dot products, softmax and temperature — with the real GPT-3 numbers worked out.
Build GPT-2 from Scratch in PyTorch: A Full Walkthrough
How to build GPT-2 from scratch in PyTorch — tokenization, causal self-attention, transformer blocks, weight tying, and a working training loop — implemented step by step and trained locally (124M parameters).