All topics

SwiGLU

Every post tagged “SwiGLU”, newest first.

1 post
LLM 11 min

Build a Mini LLM from Scratch in NumPy: RoPE, GQA, SwiGLU Visual Guide

How to build a mini LLM from scratch in NumPy — RoPE, GQA, QK-Norm, SwiGLU, tied embeddings, and a 3.87M chat companion with full architecture visuals.