Byte Pair Encoding (BPE) Explained: How GPT Tokenizers Turn Text Into Numbers
What is byte pair encoding (BPE)? Learn how GPT tokenizers split text into IDs — pretokenization, merge rules, byte-level vocab, and a from-scratch implementation you can actually debug.