Cross-Entropy Loss Explained: From Logits to LLM Training
What is cross-entropy loss? Derive softmax and negative log-likelihood, work a 3-class example, and see why MSE fails for language model classification.
What is cross-entropy loss? Derive softmax and negative log-likelihood, work a 3-class example, and see why MSE fails for language model classification.
How does a GPT actually work? A visual walkthrough of transformers: tokens, word embeddings, dot products, softmax and temperature, with real GPT-3 numbers.