Supervised fine-tuning takes a pretrained model and trains it further on curated (input, output) pairs — usually instruction-following examples. It is the cheapest, fastest way to adapt a base model to a specific behavior and precedes RLHF or DPO in every modern post-training pipeline.
- 01Distinguish SFT from pretraining.
- 02Explain why SFT dataset quality dominates dataset size.
- 03State when SFT alone suffices and when RLHF is needed.
The stage
SFT uses the same next-token-prediction loss as pretraining, but on carefully curated instruction-following data. 10,000-100,000 examples is typical; 1M is on the high end. Quality matters more than quantity — LIMA showed 1,000 good examples can beat 50,000 mediocre ones.
When it suffices
For structured tasks with clear right answers — code, classification, extraction — SFT alone is often enough. For preference-shaping — 'don't be harmful,' 'be helpful in the way our users want' — RLHF or DPO is usually necessary.
- SFT = next-token loss on curated instruction data.
- Quality dominates quantity; 1,000 good examples beat 50,000 mediocre ones.
- Zhou et al., 'LIMA: Less Is More for Alignment' (2023)