Numerical Gradient Checking: How to Debug Your Autograd Engine Before Training GPT
What is numerical gradient checking? Learn the central difference formula, why you never train with finite differences, and how to build a grad checker that catches bugs in your autograd before LayerNorm or attention.