All topics

Autograd

Every post tagged “Autograd”, newest first.

2 posts

Deep Learning 14 min

Numerical Gradient Checking: How to Debug Your Autograd Engine Before Training GPT

What is numerical gradient checking? Learn the central difference formula, why you never train with finite differences, and how to build a grad checker that catches bugs in your autograd before LayerNorm or attention.

Jun 7, 2026

Deep Learning 31 min

Backpropagation from Scratch: Build an Autograd Engine in Python

How does backpropagation work? Build a working autograd engine from scratch in ~80 lines of Python — computation graphs, the chain rule, and reverse-mode autodiff — the same core idea behind PyTorch's .backward().

May 11, 2026