DeepSeek V4 Inside: One Token Through Every Block
How does DeepSeek V4 process one token? I follow the byte p in deepseek through CSA, HCA, mHC, MoE, and MTP on a toy width, with Flash-0731 sizes.
How does DeepSeek V4 process one token? I follow the byte p in deepseek through CSA, HCA, mHC, MoE, and MTP on a toy width, with Flash-0731 sizes.
How to build a mini LLM from scratch in NumPy: RoPE, GQA, QK-Norm, SwiGLU, tied embeddings, and a 3.87M chat companion with full architecture visuals.