206Course · Practitioner
Cost and Latency
The two constraints that shape design.
The two constraints that shape design. Token economics, caching, batching, streaming, prompt caching — presented in order of impact.
- 01
Model the cost of a workflow.
- 02
Reduce latency without sacrificing quality.
- 03
Design caching intelligently.
- 04
Read a cost report and find the lever.
Prerequisites
- — System Design.
Pacing
~3 hours.
- IModule
The economics
How the numbers actually work.
- IIModule
Latency
The user-facing constraint.
A Note from the Faculty
The biggest cost lever most teams skip is caching.