Build GPT-2 from Scratch in PyTorch: A Full Walkthrough
How to build GPT-2 from scratch in PyTorch — tokenization, causal self-attention, transformer blocks, weight tying, and a working training loop — implemented step by step and trained locally (124M parameters).