Summary
The 1990s pivot from hand-coded rules to learning from data. Bayesian methods, decision trees, ensemble methods, and kernel methods dominated. The paradigm was quieter than symbolic AI's ambitions but produced systems that worked in production — credit scoring, spam filtering, search ranking.
Objectives
- 01Distinguish generative from discriminative models.
- 02Name the three dominant model families of the era.
- 03Explain why statistical methods succeeded where symbolic methods failed.
The Lesson
The paradigm shift
Instead of encoding expertise, fit a function to labeled data. The move required larger datasets, faster computers, and a willingness to accept probabilistic answers instead of proofs.
What actually shipped
Naive Bayes for spam. Random forests for tabular data. Boosted trees (AdaBoost, later XGBoost) for competitions and industry. These systems worked and still work — most non-neural production ML runs on tree ensembles.
Key Ideas
- Statistical learning traded certainty for scalability.
- Tree ensembles remain the industry default for tabular data.