Summary
An embedding maps text to a dense vector such that semantically similar text lies close in vector space. Modern embedding models (OpenAI text-embedding-3, Cohere embed-v3, open BGE) are the workhorse of retrieval systems.
Objectives
- 01State what an embedding is and what similarity means in that space.
- 02Distinguish contrastive from masked-language-model embeddings.
- 03Choose an embedding model for a specific corpus.
The Lesson
What they represent
Similarity in embedding space approximates semantic similarity. Cosine similarity is the standard metric. Dimensions are typically 512 to 3072. Higher-dimensional embeddings are more expressive but more expensive to store and search.
Key Ideas
- Embeddings encode meaning as position in vector space.
- Cosine similarity is the standard retrieval metric.