A developer searches a million-line codebase for "how we retry failed webhook deliveries." Keyword search misses files that say "exponential backoff on HTTP callbacks." An embedding model explained in plain terms: a neural network that converts text into a fixed-length numeric vector such that similar meaning yields similar vectors. Search becomes geometry: find the vectors closest to the query vector. AI code assistants embed repositories for semantic navigation; AI automation platforms embed runbooks and API docs so agents retrieve the right step before acting.
What an Embedding Represents
An embedding is a dense vector (typically 384 to 3072 dimensions) that encodes semantic and sometimes syntactic properties of a text span so that related concepts cluster in vector space. Unlike sparse bag-of-words vectors, embeddings capture paraphrase: "automobile" and "car" land near each other. The model is trained on contrastive or predictive objectives over large corpora so co-occurring or synonymous contexts align. Embeddings are not human-readable; each dimension does not map to a single feature like "pricing."
You embed at different granularities: whole documents, paragraphs, sentences, or code functions. Retrieval quality depends on matching granularity to user queries. A document-level embedding of a fifty-page manual dilutes specific facts; chunk-level embeddings improve recall for pinpoint questions at the cost of more vectors to store and search.
Cosine Similarity and Distance
Vector search ranks candidates by cosine similarity (angle between vectors) or Euclidean distance; cosine is standard because embedding models often L2-normalize outputs. Cosine similarity ranges from -1 to 1 for normalized vectors; scores above 0.75 might indicate strong match depending on model and domain. Dot product equals cosine when vectors are unit length and is faster on some hardware. Avoid comparing scores across different embedding model versions; re-embed everything when the model changes.
| Metric | When to use | Caveat |
|---|---|---|
| Cosine similarity | Default for text embeddings | Thresholds are model-specific |
| Dot product | Normalized vectors, ANN indexes | Unnormalized vectors skew results |
| Euclidean (L2) | Some multimodal models | Sensitive to magnitude |
Domain-Specific vs General Embeddings
General-purpose embedding models (OpenAI text-embedding-3, Cohere embed, open models like nomic-embed and bge) cover broad web text; domain models excel on legal, medical, scientific, or code corpora. Code search often uses code-aware embedders trained on GitHub pairs. Legal teams may fine-tune embeddings on clause libraries. Start with a strong general model and benchmark on your queries before investing in domain fine-tuning. Mismatched domains show up as missed synonyms and poor ranking on technical jargon.
Multilingual embeddings
Multilingual models map text in different languages into a shared space so a query in English can retrieve Spanish documents. Quality varies by language pair and resource prevalence. Evaluate on your actual locales; low-resource languages may need language-specific models or translation at query time.
| Model type | Best for | Tradeoff |
|---|---|---|
| General web | Support KB, mixed topics | Weaker on niche jargon |
| Code | Repos, API references | Poor on prose-only docs |
| Legal / medical | Regulated terminology | Higher licensing or training cost |
Chunk Size and Embedding Refresh
Chunk size controls what each vector represents; re-embedding (refresh) is required when source documents change or when you upgrade the embedding model. Common RAG chunks run 300 to 800 tokens with heading-aware splits. Smaller chunks improve precision; larger chunks preserve context. Parent-child patterns embed small chunks for retrieval and attach larger parent text for generation. Schedule incremental refresh on document updates; full reindex on model swaps. Stale vectors silently degrade search until users report wrong answers.
Embedding pipeline checklist
- Normalize text (encoding, boilerplate stripping, PII policy).
- Chunk with structure awareness (headings, code fences, tables).
- Attach metadata (source ID, version, ACL, language).
- Batch embed with rate-limit handling and retries.
- Upsert vectors into index; tombstone deleted sources.
- Log model name and dimension per vector for migrations.
Evaluating Embeddings with MRR and nDCG
Measure embedding quality with ranking metrics: Mean Reciprocal Rank (MRR) and normalized Discounted Cumulative Gain (nDCG) on a labeled query set where humans mark relevant documents. MRR rewards placing the first relevant result high. nDCG accounts for multiple relevant items and position discounting. Build fifty to two hundred real user queries with gold relevant chunk IDs. Compare embedding models on the same index structure before production rollout. Recall@k (whether any relevant item appears in top k) matters for RAG where the generator only sees retrieved chunks.
| Metric | What it measures | Good starting target |
|---|---|---|
| MRR | Rank of first hit | Domain-dependent; track deltas |
| Recall@10 | Any relevant in top 10 | Above 0.85 for RAG pilots |
| nDCG@10 | Ordered relevance quality | Compare models relatively |
Embedding Model Selection Criteria
Choose embedding models by benchmark on your data, supported languages, dimension and storage cost, latency, hosting (API vs self-hosted), and license terms for commercial use. Higher dimensions do not always mean better retrieval; some Matryoshka models allow truncating dimensions with modest quality loss. Self-host open weights for data residency; APIs reduce ops burden. Pair first-stage embeddings with a cross-encoder reranker when precision at top one matters.
Matryoshka and Dimension Truncation
Matryoshka embedding models train vectors so earlier dimensions carry most semantic signal, allowing truncation to 256 or 512 dimensions with smaller storage and faster search. Useful when vector database bills by RAM or when mobile clients run on-device similarity. Always re-benchmark recall@10 after truncation on your golden set; legal and medical corpora may lose more than general FAQ content. Store full-dimension vectors during experimentation, then pick the smallest dimension that meets SLA before production lock-in.
Bi-Encoder vs Cross-Encoder Reranking
Bi-encoders embed query and document separately for fast ANN search; cross-encoders jointly encode query-document pairs for accurate scoring but only at rerank depth. Production RAG almost always uses bi-encoder for first stage (top 50 to 200) and cross-encoder on top 10 to 20. Never run cross-encoder across the full corpus per query. Latency budget: bi-encoder under fifty milliseconds, cross-encoder under two hundred milliseconds for ten pairs on GPU. Open-source rerankers (bge-reranker, ms-marco MiniLM cross-encoders) close much of the gap with hosted APIs when self-hosted.
Late interaction models
ColBERT-style late interaction keeps token-level embeddings and computes fine-grained similarity at query time. Higher quality on keyword-heavy technical docs at increased storage and compute. Evaluate when vanilla bi-encoders miss exact token overlaps in API reference search.
Embedding API vs Self-Hosted
APIs (OpenAI, Cohere, Voyage, etc.) minimize ops; self-hosted open models (nomic-embed, e5, bge) maximize data control and predictable unit economics at scale. APIs charge per million tokens ingested and queried; self-hosted shifts to GPU amortization and engineer time. Regulated buyers often require embeddings inside VPC even when generation uses external LLMs. Version-pin API model names in config; providers deprecate embedding models with migration notices shorter than your reindex window.
| Deployment | Pros | Cons |
|---|---|---|
| Managed API | Fast start, auto scaling | Data egress, per-token cost |
| Self-hosted GPU | VPC control, flat GPU cost | Ops, model serving expertise |
| CPU batch overnight | Cheap bulk ingest | Slow for real-time reindex |
Common Embedding Failure Modes
Failures include wrong chunk granularity, mixed embedding model versions in one index, ignoring metadata filters, and evaluating only on easy head queries. Long documents embedded as single vectors hide interior facts. Numeric IDs and SKUs often need hybrid keyword search alongside embeddings. Monitor production click-through and thumbs-down on retrieved snippets to catch drift before aggregate MRR collapses.
Frequently Asked Questions
How many dimensions do I need?
Common ranges are 384 to 1536 dimensions; storage and index memory scale linearly while quality gains plateau. Test truncated dimensions if your vector database charges by size. Match index configuration to the model output size exactly.
Must query and document use the same model?
Yes, query and corpus vectors must come from the same embedding model and preprocessing pipeline. Mixing models breaks geometry. Some APIs offer asymmetric modes (query vs document encoders); use them only as documented by the provider.
When do keywords beat embeddings?
Exact identifiers (error codes, part numbers, statutes) often need BM25 or keyword indexes; hybrid search combines both. Embeddings win on paraphrase and conceptual similarity. Production RAG rarely relies on vectors alone.
What drives embedding cost?
Ingest cost scales with total tokens embedded; query cost scales with query volume. One-time corpus embedding dominates initial setup; incremental updates amortize. Self-hosted GPU shifts capex; APIs shift to per-token opex. Cache query embeddings for repeated dashboard filters.
How do I upgrade embedding models safely?
Run dual indexes in parallel, compare MRR on golden queries, cut over with a flag, then decommission the old index. Never mix old and new vectors in one collection. Communicate search quality changes to internal users during the transition window.
Normalization and Preprocessing Before Embed
Preprocessing changes retrieval quality as much as model choice: strip HTML boilerplate, normalize whitespace, expand acronyms consistently, and apply the same pipeline at ingest and query time. Code embeddings should preserve indentation policy (spaces vs tabs) consistently. Remove navigation chrome from scraped web pages before chunking. Hash normalized text to skip redundant re-embeds when content unchanged. Document preprocessing version in vector metadata so replays after bug fixes are traceable.
Embedding in Multimodal and Cross-Modal Search
CLIP-style models embed images and text into shared space for "find photos like this caption" workflows; audio and video pipelines embed transcript segments similarly. Cross-modal search powers media libraries in video AI products. Dimension and model differ from text-only stacks; do not mix CLIP vectors with text embedding indexes without a fusion layer. Evaluate zero-shot retrieval on representative thumbnails and captions before promising semantic video search in marketing copy.
Synthetic Queries for Embedding Eval
When golden user queries are scarce, generate synthetic questions from document chunks with an LLM, then human-filter for quality before using them in MRR benchmarks. Synthetic queries bootstrap eval but overfit to generator phrasing; refresh with real search logs monthly. Pair synthetic with at least fifty hand-written questions from domain experts. Track metric divergence between synthetic and organic query sets to catch eval blind spots before launch.
Privacy and PII in Embedding Pipelines
Embeddings can leak semantic information about sensitive text; treat embed API calls like data processing events subject to retention and residency policies. Redact or tokenize PII before embed when regulations require. Some teams embed only after automated classification marks chunks as non-sensitive. Query logs that store raw user questions alongside vectors need the same retention limits as chat logs. On-prem embed servers reduce third-party exposure but shift security burden to your team. Document subprocessors in privacy policies when using hosted embedding APIs.
Conclusion
Embedding models are the search layer behind semantic retrieval, RAG, and many automation workflows. Text becomes vectors; cosine similarity finds neighbors; chunking and refresh policies keep the index honest. Benchmark with MRR and nDCG on your queries, pick general or domain models deliberately, and hybridize with keywords where exact matches matter. Treat embedding choice as infrastructure with versioned migrations, not a one-time setup step.
