Convolutional Neural Networks: CNN Math Explained
How do convolutional neural networks work? CNN math explained with kernels, padding, stride, pooling, receptive fields, and a full backpropagation example.
Looped Transformers Explained: Recurrent Depth and Astra
What is a looped transformer? How recurrent depth reuses layers, why Nanbeige 4.2 runs a 22-layer stack twice, and why that does not hide chain of thought.
GPT Math Explained: The Full Forward Pass Beyond Attention
How does GPT math work end to end? One training step from token IDs through Q/K/V, attention, MLP, logits, cross-entropy loss, backprop, and AdamW.
Cross-Entropy Loss Explained: From Logits to LLM Training
What is cross-entropy loss? Derive softmax and negative log-likelihood, work a 3-class example, and see why MSE fails for language model classification.
Adam and AdamW Explained: How LLMs Update Their Weights
How does the Adam optimizer work? Derive momentum, RMSprop and bias correction, see why AdamW decouples weight decay, and walk a numeric update by hand.
Numerical Gradient Checking: Debug Your Autograd Engine
What is numerical gradient checking? The central difference formula, why you never train with finite differences, and a grad checker that catches real bugs.
Transformer from Scratch: Forward Pass and Backprop by Hand
One full transformer training step worked by hand: embeddings, positional encoding, attention, layer norm, cross-entropy loss, backprop, and Adam.
How Attention Works in Transformers: Queries, Keys, Values
How does attention work in transformers? Queries, keys, values, the attention pattern, masking, and multi-head attention, with real GPT-3 parameter counts.
How GPT Actually Works: A Visual Guide to Transformers
How does a GPT actually work? A visual walkthrough of transformers: tokens, word embeddings, dot products, softmax and temperature, with real GPT-3 numbers.
Backpropagation from Scratch: Build an Autograd Engine
How does backpropagation work? Build a working autograd engine from scratch in ~80 lines of Python: computation graphs, chain rule, reverse-mode autodiff.