Reading
Andrej Karpathy, 'Let's build GPT: from scratch, in code, spelled out,' YouTube (2023).
The clearest end-to-end walkthrough of a transformer language model in existence. Two hours; produces a working GPT in ~300 lines of PyTorch. Watch it once with a notebook open, once without. There is no substitute for having built one.
Reading
Primary and secondary sources for this lesson.