All topics

NLP

Every post tagged “NLP”, newest first.

2 posts
Embeddings 15 min

Token Embeddings Explained: How LLMs Turn Token IDs Into Meaning Vectors

What is a token embedding? How LLMs map token IDs to learned vectors — the embedding matrix, gather lookup vs one-hot matmul, parameter counts, and a from-scratch NumPy implementation.

Tokenization 21 min

Byte Pair Encoding (BPE) Explained: How GPT Tokenizers Turn Text Into Numbers

What is byte pair encoding (BPE)? Learn how GPT tokenizers split text into IDs — pretokenization, merge rules, byte-level vocab, and a from-scratch implementation you can actually debug.