Holding the LLM Stack in Your Head

Nick Gustafson's series walks the full modern LLM stack, bottom to top: from the linear algebra inside a single attention head through training, inference engines, retrieval, and the agent protocols expected in 2026. It spans ten arcs and roughly eighty posts, ordered by dependency rather than topic popularity.

The author wrote it as a learning exercise in collaboration with Claude Opus 4.8 and describes it as "thinking out loud with a model" — not an authoritative reference. He advises readers to double-check anything they rely on.

The ten arcs cover mathematical prerequisites (vectors, softmax, cross-entropy, gradients, optimizers, GPU numerics), pre-transformer language modeling, tokenization, transformers from first principles, decoding and inference algorithms, inference serving systems, training and post-training, evaluation methodology, retrieval and context engineering, and finally tools, protocols, and agent loops. The series also includes curated learning paths for understanding attention, diagnosing inference latency, building with RAG, and building agents.

The series is a full first draft, with polish and corrections in progress.