102Course · Foundation
How LLMs Work
The mechanism, in institutional detail.
The mechanism, in institutional detail. Tokens, embeddings, the transformer block, training, RLHF, DPO, inference — enough depth that the Scholar can diagnose a failure from its symptoms rather than from folklore.
This is the course that separates operators from users.
- 01
Read the transformer architecture end to end.
- 02
Explain attention at the level of Q, K, V.
- 03
Distinguish training from inference from sampling.
- 04
Diagnose a failure mode from its symptoms.
Prerequisites
- — Comfort with linear algebra vocabulary.
Pacing
~6 hours.
- IModule
Tokens and representations
Where the model begins.
- IIModule
The transformer block
Precisely.
- IIIModule
Training
How weights actually get set.
- IVModule
Inference
What happens when you talk to a model.
- VModule
What models can and cannot do
The honest limits.
A Note from the Faculty
Read Attention Is All You Need. Then read it again in three months.