All topics

Embeddings

Every post tagged “Embeddings”, newest first.

2 posts

Embeddings 15 min

Token Embeddings Explained: How LLMs Turn Token IDs Into Meaning Vectors

What is a token embedding? How LLMs map token IDs to learned vectors — the embedding matrix, gather lookup vs one-hot matmul, parameter counts, and a from-scratch NumPy implementation.

Jun 10, 2026

Positional Encoding 12 min

Positional Encoding Explained: How Transformers Learn Word Order

What is positional encoding in transformers? Why attention is order-blind, learned vs sinusoidal position embeddings, why we add instead of concatenate, and a from-scratch NumPy implementation.

Jun 10, 2026