What Embedding Models Actually Do (and Why the Definition Matters)
Embedding models take a piece of text and return a fixed-length vector of floats. That's the whole trick. A sentence like "the turbine failed at high RPM" becomes something like [-0.023, 0.841, 0.012, ...] with 768 or 1536 dimensions, and semantically similar sentences land close together in that space. Every RAG pipeline and semantic search system we've shipped in the last two years depends on this machinery working correctly.
The problem is that "working well" is context-dependent in ways that catch people off guard. I've watched students in our NLP & LLM Engineering course pick the top model from the MTEB leaderboard, plug it into their RAG stack, and get retrieval precision that's 15 points worse than what a smaller, older model gave them on their specific corpus. Domain matters. A lot.
How to Pick an Embedding Model Without Guessing
Start with constraints, not benchmarks. Before you open the MTEB leaderboard, answer two questions: Does your data stay inside your infrastructure? What's your p95 latency budget for the embedding call?
If the data has to stay on-prem, or you need sub-50ms embedding latency, you're looking at self-hosted open-source. If you can tolerate API calls and the data isn't sensitive, OpenAI text-embedding-3-large or Cohere embed-v3 are legitimate options with almost no setup friction. Both are fine. Neither is obviously better until you benchmark.
For open-source, the models worth testing right now are bge-large-en-v1.5 (768 dims, excellent retrieval on most English corpora) and E5-mistral-7b-instruct (4096 dims, expensive to run but genuinely strong on complex queries). The newer gte-Qwen2-7B-instruct is also worth a slot. Don't overlook mxbai-embed-large-v1 either. It punches above its weight on retrieval tasks and runs comfortably on the LLM Inference (L4) lab environment without needing a massive GPU allocation.
Once you have three or four candidates, benchmark them on your actual data. Not MTEB. Pull 200 to 300 real queries from your system logs or from your users. Grab the documents they should retrieve. Compute precision@5 and NDCG@10 for each model. This takes a few hours, and it will change your rankings relative to MTEB almost every time.
How to Evaluate Embeddings on Your Own Domain
Evaluating embeddings without a domain test set is the most common mistake we see in capstone reviews. Engineers grab a model, run a few manual spot-checks, decide it "feels right," and ship it. That's fine until a user asks something slightly outside the obvious phrasing and retrieval collapses.
Here's a minimal evaluation setup that actually works:
from sentence_transformers import SentenceTransformer, util
import numpy as np
model = SentenceTransformer("BAAI/bge-large-en-v1.5")
# queries: list of strings
# corpus: list of strings
# relevant_docs: dict mapping query index -> list of relevant corpus indices
corpus_embeddings = model.encode(corpus, normalize_embeddings=True)
query_embeddings = model.encode(queries, normalize_embeddings=True)
hits = util.semantic_search(query_embeddings, corpus_embeddings, top_k=5)
precision_scores = []
for i, query_hits in enumerate(hits):
retrieved = [h["corpus_id"] for h in query_hits]
relevant = relevant_docs[i]
precision = len(set(retrieved) & set(relevant)) / len(retrieved)
precision_scores.append(precision)
print(f"Mean Precision@5: {np.mean(precision_scores):.4f}")
Run this for each candidate. The differences will surprise you. In office hours last March, a student found that bge-large-en-v1.5 beat OpenAI text-embedding-3-large by 11 precision points on internal engineering ticket data. The tickets were full of internal acronyms and product names the OpenAI model had never seen in meaningful context. No amount of MTEB scrolling would have told her that.
The sentence transformers library makes this kind of evaluation fast to set up. That's why it's still our default for benchmarking even when the production model ends up being something else entirely.
When OpenAI Embeddings Win (and When They Don't)
This isn't a philosophical debate. It's an engineering tradeoff with actual numbers attached.
OpenAI text-embedding-3-large costs $0.00013 per 1K tokens. Running bge-large-en-v1.5 on a spot L4 instance runs around $0.0001 per 1K tokens in compute, and drops further at volume. At 100 million tokens a month, that gap is real money. Not abstract money. Rent money.
OpenAI wins on zero infrastructure overhead and strong out-of-the-box performance on general English. The 3072-dimension variant gives you headroom for complex similarity tasks. If you're building a prototype or an internal tool where your team isn't scaling to millions of queries, just use the API and move on. Seriously.
Open-source wins on latency-sensitive applications (API round-trip adds 80 to 200ms depending on load), regulated industries where data can't leave your network, and any domain where you plan to fine-tune anyway. If you're going to retrain the model, you need a model you can actually retrain. That rules out the API.
When Should You Fine-Tune an Embedding Model?
Fine-tuning is the right call when two things are both true: your domain has vocabulary or sentence structure that's genuinely out-of-distribution for general models, and you've already tried two or three top-tier base models without getting above your precision threshold. Not one model. Two or three, benchmarked properly.
In our NLP & LLM Engineering course we walk through fine-tuning with the Sentence Transformers library using MultipleNegativesRankingLoss. You need query-document pairs where you know which documents are relevant. 500 pairs will get you something useful. 2,000 to 5,000 pairs will get you something good.
from sentence_transformers import SentenceTransformer, InputExample, losses
from torch.utils.data import DataLoader
model = SentenceTransformer("BAAI/bge-large-en-v1.5")
train_examples = [
InputExample(texts=["query text", "relevant document text"]),
# ... more pairs
]
train_dataloader = DataLoader(train_examples, shuffle=True, batch_size=16)
train_loss = losses.MultipleNegativesRankingLoss(model)
model.fit(
train_objectives=[(train_dataloader, train_loss)],
epochs=3,
warmup_steps=100,
output_path="./my-finetuned-embedder"
)
We run these jobs in the LLM Fine-Tuning (A100) lab environment. On an A100 80GB with a 2,000-pair dataset, a three-epoch run on bge-large takes under 20 minutes. Fast enough that there's no excuse not to iterate.
If you're in healthcare or legal and your retrieval precision@5 is below 0.75, fine-tuning is almost always worth the investment. Generic embedding models haven't seen your discharge summaries or your contract clauses with enough frequency to cluster them well. Our AI for Healthcare & Life Sciences course covers exactly this scenario, including how to build labeled pairs from expert annotations when you don't have existing query logs.
One thing I'd push back on hard: don't fine-tune as your first move. Benchmark first. You might find that gte-Qwen2-7B-instruct already handles your domain well enough, and fine-tuning a 7B model is a non-trivial infrastructure commitment. Run the evaluation script above on your real data before you touch a training loop. Most of the time, the bottleneck isn't the embedding model anyway. It's the chunking strategy.
