502Course · Expert
Mixture of Experts
Sparsity, worked.
- 01
Explain routing.
- 02
Read a MoE paper.
- 03
Understand training instabilities.
- 04
Estimate MoE cost.
- IModule
The architecture
How MoE works.
Sparsity, worked.
Explain routing.
Read a MoE paper.
Understand training instabilities.
Estimate MoE cost.
How MoE works.