What is semantic search? Semantic vs keyword search
Semantic search finds results by meaning using embeddings, not shared words. How it differs from keyword (BM25) search, where each fails, with a Python demo.

Semantic search finds documents by meaning rather than by matching words: it turns the query and every document into embedding vectors and returns the documents whose vectors sit closest to the query’s. A search for “forgot my password” can find a page titled “reset your login credentials” even though they share no words. Keyword search, usually BM25, does the opposite: it ranks documents by the exact terms they share with the query, which makes it precise on names and codes and blind to paraphrase.
How does semantic search work?
There are two phases, and both use the same embedding model.
Indexing. Split each document into chunks, run every chunk through an embedding model, and store the vectors in a vector database or a vector column.
Querying. Embed the user’s query with the same model, then find the stored vectors nearest to it, usually by cosine similarity. Return the top K chunks.
The model is doing the semantic part. It was trained so that texts with related meaning get nearby vectors. Sentence-BERT (Reimers and Gurevych, EMNLP 2019) is the paper that made this practical for sentences: rather than feeding every pair of sentences through BERT together, it embeds each sentence once and compares vectors. Their example is finding the most similar pair in 10,000 sentences, which took about 65 hours the old way and about 5 seconds with SBERT embeddings.
How does keyword search work?
Keyword search builds an inverted index: for every term, the list of documents containing it. At query time it scores documents that contain query terms, and the standard scoring function is BM25, set out in Robertson and Zaragoza’s The Probabilistic Relevance Framework: BM25 and Beyond (2009). BM25 rewards a term more when it is rare across the corpus, gives diminishing returns for repeating a term, and normalizes for document length. Anthropic’s contextual retrieval post describes it as building on TF-IDF with length normalization and a saturation function on term frequency. I break it down fully in what is BM25.
Semantic search vs keyword search: what is the difference?
| Keyword (BM25) | Semantic (embeddings) | |
|---|---|---|
| Matches on | shared terms | nearby meaning |
| Paraphrase and synonyms | misses them | finds them |
| Exact codes, names, SKUs | strong | often weak |
| Needs a model | no | yes, at index and query time |
| Explaining a result | easy: these words matched | hard: the vectors were close |
| Out-of-domain text | holds up well | depends on the model |
The cleanest way to see the trade-off is to run both on the same documents.
What does semantic vs keyword search look like in code?
This script ranks five support articles with BM25 and with vectors, then fuses the two lists. The “embedding model” is a hand-written table that maps words onto four concepts (access, money, failure, shipping), so it runs offline with no packages. A real model learns hundreds of such directions from data; the mechanics are the same.
# semantic.py: keyword search (BM25) vs semantic search (vectors), on the same five docs.
# Python 3 stdlib only. The "embedding model" is a hand-written word -> concept table so the
# example runs offline; a real model learns these directions from data.
import math, re
docs = {
"d1": "How to reset your login credentials if you cannot sign in",
"d2": "Refund policy: we return payments within 7 days",
"d3": "E-4012: the webhook signature did not match the payload",
"d4": "Common errors when calling the API and how to fix them",
"d5": "Track your parcel and change the delivery address",
}
CONCEPTS = ["access", "money", "failure", "shipping"]
TABLE = { # word -> weight on each concept (access, money, failure, shipping)
"password": (1, 0, 0, 0), "login": (1, 0, 0, 0), "credentials": (1, 0, 0, 0),
"sign": (.6, 0, 0, 0), "reset": (.7, 0, .2, 0), "forgot": (.6, 0, .3, 0),
"refund": (0, 1, 0, 0), "payments": (0, 1, 0, 0),
"error": (0, 0, 1, 0), "errors": (0, 0, 1, 0), "fix": (0, 0, .7, 0), "failed": (0, 0, 1, 0),
"parcel": (0, 0, 0, 1), "delivery": (0, 0, 0, 1),
}
def tokens(s): return re.findall(r"[a-z0-9-]+", s.lower())
def embed(s):
v = [0.0] * len(CONCEPTS)
for t in tokens(s):
for i, w in enumerate(TABLE.get(t, (0, 0, 0, 0))):
v[i] += w
n = math.sqrt(sum(x * x for x in v)) or 1.0
return [x / n for x in v]
def bm25(query, k1=1.2, b=0.75):
toks = {d: tokens(t) for d, t in docs.items()}
avg = sum(map(len, toks.values())) / len(toks)
scores = {}
for d, ts in toks.items():
s = 0.0
for q in set(tokens(query)):
df = sum(q in t for t in toks.values())
if df == 0: continue
idf = math.log(1 + (len(toks) - df + 0.5) / (df + 0.5))
tf = ts.count(q)
s += idf * tf * (k1 + 1) / (tf + k1 * (1 - b + b * len(ts) / avg))
scores[d] = s
return scores
def dense(query):
q = embed(query)
return {d: sum(a * b for a, b in zip(q, embed(t))) for d, t in docs.items()}
def top(scores, n=3, fmt=".2f"):
hits = [(d, s) for d, s in sorted(scores.items(), key=lambda x: -x[1]) if s > 0][:n]
return " ".join(f"{d}:{s:{fmt}}" for d, s in hits) or "(no matches)"
def rrf(*rankings, k=60):
fused = {}
for scores in rankings:
hits = [d for d in sorted(scores, key=lambda d: -scores[d]) if scores[d] > 0]
for rank, d in enumerate(hits, 1):
fused[d] = fused.get(d, 0) + 1 / (k + rank)
return fused
for q in ["forgot my password", "E-4012 failed"]:
print(f"query: {q!r}")
print(" keyword (BM25) ", top(bm25(q)))
print(" semantic (cos) ", top(dense(q)))
print(" hybrid (RRF) ", top(rrf(bm25(q), dense(q)), fmt=".4f"))
Output (Python 3.9.6):
query: 'forgot my password'
keyword (BM25) (no matches)
semantic (cos) d1:0.99 d4:0.18
hybrid (RRF) d1:0.0164 d4:0.0161
query: 'E-4012 failed'
keyword (BM25) d3:1.41
semantic (cos) d4:1.00 d1:0.06
hybrid (RRF) d3:0.0164 d4:0.0164 d1:0.0161
The first query is the case semantic search was built for. “forgot my password” shares no terms with any document, so BM25 returns nothing. The vector side maps “forgot” and “password” onto the access concept and finds d1, the credentials article, with a score of 0.99.
The second query flips it. “E-4012” is an identifier. BM25 finds d3 immediately because that exact token appears in one document and nowhere else, which gives it a high inverse document frequency. The vector side has no idea what E-4012 is, latches onto “failed”, and ranks the generic errors page first. The document the user needed is not in its list at all.
The hybrid line fuses both rankings with reciprocal rank fusion (each list contributes 1 / (60 + rank) per document). For the first query it keeps the semantic winner on top. For the second it puts the exact match back in the top two, tied with the generic page. That is the argument for hybrid search in one screen, and reciprocal rank fusion gets its own post.
Is semantic search always better than keyword search?
No, and the research is consistent on this. Dense Passage Retrieval (Karpukhin et al., EMNLP 2020) beat a Lucene BM25 system by 9 to 19 points of top-20 accuracy on open-domain question answering, in the domains it was trained for. The BEIR benchmark (Thakur et al., NeurIPS 2021) then tested ten retrieval systems on 18 datasets outside their training data. Its abstract calls BM25 a strong baseline and reports that dense retrievers, while efficient, often underperform out of domain, with rerankers and late-interaction models such as ColBERT doing best at a higher compute cost.
Embedding models have improved a lot since 2021, and the MTEB leaderboard is where to compare current ones. But the structural point stands: an embedding compresses a passage into one point, and rare identifiers, part numbers and names do not survive that compression well. Anthropic’s contextual retrieval write-up uses exactly this example, a query for “Error code TS-999” where embeddings return general error-code content and miss the exact match. In their tests on a 5.7% baseline failure rate for top-20 retrieval, contextual embeddings cut failures by 35%, and adding contextual BM25 on top cut them by 49%.
When should I use semantic search?
- Use semantic search when users describe what they want in their own words: support questions, natural-language questions over documentation, “find me something like this”, cross-lingual search with a multilingual model.
- Use keyword search when the query contains things that must match exactly: product codes, error strings, legal citations, names, file paths.
- Use both when you can’t predict which kind of query will arrive, which is almost always. Run BM25 and vector search in parallel, fuse the lists, and if quality matters, rerank the top candidates with a cross-encoder.
What I would do
The search product I built, rSearch, retrieves from the web through a search API, so the keyword-versus-vector decision is made upstream of my code. For a private corpus that decision is yours, and this is how I would make it:
- Start with BM25. It needs no model, it is explainable, and it gives you a baseline to beat.
- Write 50 to 100 real queries with their correct documents before you add embeddings. Without them, “semantic search feels better” is an anecdote.
- Add vector search beside BM25, not instead of it, and fuse with reciprocal rank fusion. Measure recall@10 for BM25 alone, vectors alone and the fusion.
- Add a reranker over the fused top 50 if the top three results still miss.
- Log the queries where both retrievers fail. They tell you whether the problem is chunking, the embedding model or content that does not exist.
FAQ
What is the difference between semantic search and vector search?
Vector search is the mechanism: nearest-neighbor lookup over vectors. Semantic search is the use: finding results by meaning. Semantic search is usually implemented with vector search over text embeddings.
Is semantic search the same as RAG?
No. Semantic search returns relevant documents. RAG (retrieval-augmented generation) uses a retrieval step, often semantic or hybrid search, to put those documents into a language model’s prompt so it can answer from them.
Does Google use semantic search?
Google does not publish its ranking system in full, so I won’t describe it. The ColBERT paper (2020) notes that BERT-based rankers had by then been adapted for production by both Google and Bing, citing each company’s own announcement. That is learned models on top of a large retrieval system, which is the same layered pattern as here at a much larger scale.
Why does semantic search return irrelevant results?
Usually because similar is not the same as relevant. An embedding puts “how to enable 2FA” close to “how to disable 2FA”, since both are about two-factor settings. A reranker, a keyword signal or a filter is what separates them.
Sources
- Reimers and Gurevych, Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks, EMNLP 2019
- Karpukhin et al., Dense Passage Retrieval for Open-Domain Question Answering, EMNLP 2020
- Thakur et al., BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models, NeurIPS 2021 Datasets and Benchmarks
- Robertson and Zaragoza, The Probabilistic Relevance Framework: BM25 and Beyond, Foundations and Trends in Information Retrieval, 2009
- Anthropic, Introducing Contextual Retrieval, September 19, 2024
- Khattab and Zaharia, ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT, SIGIR 2020