How does attention work in transformers? A visual, step-by-step explanation of the attention mechanism — queries, keys, values, the attention pattern, masking, the low-rank value trick, and multi-head attention — with the real GPT-3 parameter counts.
How does a GPT actually work? A plain-English, visual walkthrough of transformers — tokens, word embeddings, why directions in vector space mean something, dot products, softmax and temperature — with the real GPT-3 numbers worked out.
A complete intuitive introduction to vectors, matrices, dot products, determinants, eigenvectors, basis changes, and how linear algebra powers machine learning.