microllm · curriculum

From "what is a language model" to a tiny GPT that writes game characters — each lesson is a thing you poke at. ELI5 on the maths, deep on the CS.

Module 0 · Intuitions (no jargon, all interactive)

00 What a language model does
It guesses the next word. A live next-word predictor + temperature. live 01 Knobs & loss
A machine full of knobs, and one number that says how wrong it is. Drag two knobs to fit a line. live 02 Rolling downhill (gradient descent)
Training = nudge every knob downhill. Watch a ball find the valley; break it with a big learning rate. live 03 One neuron, by hand
A sigmoid neuron learns a function from examples — loss + gradient descent in the smallest real net. live 04 Autograd
Gradients for free. Step a backward pass through the computation graph and watch the slopes flow. live

Module 1 · Text becomes numbers

05 Tokenization
Type a sentence, watch it shatter into tokens and ids. Char-level vs BPE. soon 06 Embeddings
Every token becomes a point in space; similar words drift together. soon

Module 2 · The one big idea — attention

07 Bigram vs. context
Why "last word only" isn't enough, and what breaks. soon 08 Self-attention
Each word looks at the others. The attention heatmap, live. soon 09 The transformer block
Multi-head + MLP + residual + layernorm = a GPT layer. soon

Module 3 · Build & train the real thing

10 Assemble the GPT
Stack the blocks. The whole architecture on one screen. soon 11 Train on TinyStories
Watch the loss fall; read what it writes at each checkpoint. soon 12 Sampling — temperature & top-k
The knobs that turn probabilities into prose. soon

Module 4 · Into the game

13 Finetune for characters & quests
Bend the model to your game's format. soon 14 Run it in the browser
Export to ONNX; generate content live inside your WebGL game. soon