202Course · Practitioner
Evaluation
The one discipline that separates professionals.
The one discipline that separates the professionals. Offline evals, online evals, metric design, and the specific ways LLM-as-judge is right or a shortcut.
- 01
Design a defensible eval.
- 02
Distinguish offline from online evaluation.
- 03
Read an eval report.
- 04
Detect eval malpractice.
Prerequisites
- — System Design.
Pacing
~4 hours plus the Assignment.
- IModule
Offline evals
Test sets and their construction.
- IIModule
Online evals
In production.
A Note from the Faculty
Every serious AI team has an eval before it has a model.