Summary
The system prompt is a special message with elevated instructions. Its actual effect varies: strong on OpenAI and Anthropic, modest on some open models. The mythology around system prompts far exceeds their empirical power.
Objectives
- 01State what the system prompt actually is (a message with a role).
- 02Distinguish provider treatments.
- 03Recognize when a system prompt is being asked to do too much.
The Lesson
The reality
In chat-completion APIs, 'system' is one of several roles a message can have. The model has been trained to weight system-role instructions strongly. On strong models this weighting is robust; on weaker or older models it is easily overridden by the user turn.
What to put there
Role, format, constraints, tone, refusal criteria. Not one-shot examples (those go in the conversation). Not per-query context (put that in the user turn). Keep system prompts under ~500 tokens where possible — long system prompts dilute their own signal.
Key Ideas
- System prompts are a special role, not magic.
- Short and pointed beats long and comprehensive.