Raw capture — raw/learn-ai-layer-by-layer.md

url: https://learnai.robennals.org/ date_fetched: 2026-07-05 backfilled: true

Learn AI Layer by Layer

An interactive guide to understanding AI from first principles.

Subscribe to messyprogress.substack.com to find out when new chapters get released.

Introduction

What this is and how to use it

Everything Is Numbers

Computation

The Power of Incremental Improvement

Optimization

Building a Brain

Neural networks

Describing the World with Numbers

Vectors

From Words to Meanings

Embeddings

Understanding by Predicting

Next-word prediction

Paying Attention

Attention

Where Am I?

Positional encoding

One Architecture to Rule Them All

Transformers

Thinking by Rotating

Matrix math

Coming soon

Why Training Almost Doesn't Work

Making training work

Coming soon

Only Wake the Specialists You Need

Mixture of experts

Coming soon

Remembering a Million Words

Long context

Coming soon

Running Models Fast

Inference and hardware

Coming soon

Looking Inside the Mind

Interpretability

Coming soon

Learning from Experience

Reinforcement learning

Coming soon

Getting Better by Beating Yourself

Self-play

Coming soon

Thinking by Talking to Yourself

Reasoning models

Coming soon

Teaching AI Right from Wrong

Alignment

Coming soon

Models Teaching Models

Distillation and synthetic data

Coming soon

Teaching Machines to See

Image comprehension

Coming soon

Drawing Pictures

Image generation

Coming soon

Simulating Reality

World models

Coming soon

Listening, Speaking, and Singing

Audio

Coming soon

Getting Things Done

Agents and tool use

Coming soon

Making Stuff Up

Hallucination and grounding

Coming soon

Getting the Right Information

Context management

Coming soon