All topics

Softmax

Every post tagged “Softmax”, newest first.

2 posts
Deep Learning 9 min

Cross-Entropy Loss Explained: From Logits to Language Model Training

What is cross-entropy loss? Derive softmax + negative log-likelihood step by step, work a 3-class numeric example, see why MSE fails for classification, and build the loss GPT training actually uses.
Transformers 11 min

How GPT Actually Works: A Visual Guide to Transformers

How does a GPT actually work? A plain-English, visual walkthrough of transformers — tokens, word embeddings, why directions in vector space mean something, dot products, softmax and temperature — with the real GPT-3 numbers worked out.