All topics

PyTorch

Every post tagged “PyTorch”, newest first.

2 posts

GPT-2 40 min

Build GPT-2 from Scratch in PyTorch: A Full Walkthrough

How to build GPT-2 from scratch in PyTorch — tokenization, causal self-attention, transformer blocks, weight tying, and a working training loop — implemented step by step and trained locally (124M parameters).

May 11, 2026

Deep Learning 31 min

Backpropagation from Scratch: Build an Autograd Engine in Python

How does backpropagation work? Build a working autograd engine from scratch in ~80 lines of Python — computation graphs, the chain rule, and reverse-mode autodiff — the same core idea behind PyTorch's .backward().

May 11, 2026