Running models locally trades API convenience for control. Privacy (data does not leave your network), cost at high volume (no per-token charges), latency (no network round-trip), and sovereignty (immune to provider changes) are the cases. Peak capability and operational simplicity are what you give up.
- 01State the four cases where local wins.
- 02State the three cases where API wins.
- 03Recognize the honest capability gap.
When local wins
1. Sensitive data (health, legal, defense) with strict data-residency requirements. 2. Very high volume (millions of daily calls) where amortized GPU cost beats API markup. 3. Sub-second latency requirements not achievable over public internet. 4. Sovereignty (running when the provider is down or the relationship is over).
When API wins
1. Frontier capability (Llama 70B is not GPT-4o). 2. Multimodal, especially audio/video. 3. Small teams without ops capacity. 4. Bursty workloads where GPU idle time is expensive.
- Local is a control decision, not a cost decision.
- Peak capability lives at the API frontier; local is 12-24 months behind.