What are Matryoshka embeddings? Shorter vectors, same model
Matryoshka embeddings pack the most useful information into the first dimensions, so you can truncate vectors to cut storage. How it works, with a Python demo.

Matryoshka embeddings are embedding vectors trained so that their first dimensions carry the most important information, which lets you cut a vector down to its first 256 or 64 numbers and still use it for search. Like the Russian nesting dolls the method is named after, each shorter prefix is a smaller, coarser copy of the full embedding. The technique comes from Matryoshka Representation Learning (MRL), a NeurIPS 2022 paper by Aditya Kusupati and colleagues at the University of Washington, Google Research and Harvard.
Why would I want shorter embeddings?
Because every dimension costs storage, memory and compute on every query. A 3,072-dimension float32 vector is 12 KB. Ten million of them is about 123 GB before any index overhead, and an HNSW index wants that in RAM. Halve the dimensions and you halve the raw storage and roughly halve the work per distance computation.
The old way to get a smaller vector was to train a separate smaller model or to run a post-hoc compression step such as PCA. MRL’s pitch, in the paper’s words, is one embedding that “can adapt to the computational constraints of downstream tasks” with no extra cost at inference. You embed once at full size, and decide later, per use case, how much of it to keep.
How does Matryoshka Representation Learning work?
A normal embedding model is trained with one loss on the full vector. Nothing forces any particular dimension to matter more than another, and the paper notes that deep learning models “tend to diffuse information across the entire representation vector”. Cut such a vector in half and you throw away half of a smeared signal.
MRL changes only the loss. It picks a set of nested sizes, for example {8, 16, 32, ..., 1024, 2048} in the paper’s ImageNet setup, and computes the training loss on each prefix of the vector: the first 8 numbers, the first 16, and so on up to the full size. The losses are added together (the paper uses equal weights) and the model is trained on the sum. To score well at size 64, the model has to put its most useful information in the first 64 numbers; to score well at 2,048, it has to keep refining with the rest. Because the set of sizes is logarithmic, the paper calls the overhead minimal.
The Hugging Face Matryoshka write-up (February 2024) shows the same recipe for text in Sentence Transformers: wrap a normal loss in MatryoshkaLoss with dimensions such as 768, 512, 256, 128 and 64, and “the loss values for each dimensionality are added together”.
How much quality do Matryoshka embeddings lose when truncated?
Less than you would expect, if the model was trained for it. The MRL paper reports, on ImageNet-1K, up to 14 times smaller embeddings at the same classification accuracy, and for retrieval, a shortlist-then-rerank scheme with up to 128 times fewer theoretical FLOPs and 14 times wall-clock speedups at comparable accuracy.
For text, the Hugging Face post trained a Matryoshka model and a standard one on the same data and evaluated on STSBenchmark. At 64 of 768 dimensions (8.3% of the size), the Matryoshka model kept 98.37% of its full-size performance, against 96.46% for the standard model. That gap looks small on one benchmark; on a retrieval task with millions of near-duplicates, two points of similarity quality can move a lot of rankings.
What does truncation look like in code?
I can’t train a real model in the standard library, so this is a simulation. Both vector sets below are built from the same 24 hidden topics and differ in one thing: in the “even” set the information is smeared across all 256 dimensions, as in an ordinary model, and in the “front” set the strong directions are concentrated in the early dimensions, which is the property MRL training produces. Then I truncate and measure how much of the full-size top 10 survives.
# matryoshka.py: why truncating an embedding only works if the model front-loads information.
# Simulation in Python 3 standard library only. No real model: we build two kinds of
# 256-dim vectors from the same 24 hidden "topics" and compare truncation.
import math, random
random.seed(3)
D, N, TOPICS = 256, 2000, 24
def unit(v):
n = math.sqrt(sum(x * x for x in v)); return [x / n for x in v]
def make(spread):
"""Mix hidden topic weights into D dims. spread='front' puts the strong
directions first (what Matryoshka training aims for); 'even' smears them."""
basis = []
for t in range(TOPICS):
if spread == "front":
scale = [math.exp(-d / 24) for d in range(D)] # energy decays with index
else:
scale = [1.0] * D
basis.append([random.gauss(0, 1) * s for s in scale])
def embed(weights):
return [sum(w * basis[t][d] for t, w in enumerate(weights)) for d in range(D)]
return embed
latent = [[random.gauss(0, 1) for _ in range(TOPICS)] for _ in range(N)]
qlat = [[x + random.gauss(0, 0.3) for x in latent[i]] for i in random.sample(range(N), 40)]
def top10(qs, vs):
out = []
for q in qs:
out.append(set(sorted(range(N), key=lambda j: -sum(a * b for a, b in zip(q, vs[j])))[:10]))
return out
for spread in ("even", "front"):
embed = make(spread)
docs = [unit(embed(w)) for w in latent]
qs = [unit(embed(w)) for w in qlat]
full = top10(qs, docs)
line = []
for k in (16, 32, 64, 128):
cut_d = [unit(v[:k]) for v in docs] # truncate, then re-normalize
cut_q = [unit(v[:k]) for v in qs]
got = top10(cut_q, cut_d)
overlap = sum(len(a & b) for a, b in zip(got, full)) / (10 * len(full))
line.append(f"{k:>3}d {overlap:.2f}")
print(f"{spread:>5}-spread, top-10 overlap with full 256d: " + " ".join(line))
length = math.sqrt(sum(x * x for x in docs[0][:32]))
print(f" a unit vector cut to 32d has length {length:.3f} until you re-normalize")
Output (Python 3.9.6):
even-spread, top-10 overlap with full 256d: 16d 0.30 32d 0.44 64d 0.60 128d 0.75
a unit vector cut to 32d has length 0.375 until you re-normalize
front-spread, top-10 overlap with full 256d: 16d 0.64 32d 0.89 64d 0.99 128d 1.00
a unit vector cut to 32d has length 0.949 until you re-normalize
With smeared information, keeping half the vector (128 of 256) recovers only 75% of the full-size top 10, and a quarter recovers 60%. With front-loaded information, a quarter of the vector (64 dimensions) recovers 99%. The numbers are a property of my synthetic data, not of any real model, but the shape matches what the papers report: truncation is close to free for a model trained for it and expensive for one that wasn’t.
The length lines are the practical warning. A truncated unit vector is no longer unit length. If your search uses the dot product on the assumption that vectors are normalized (see cosine similarity), the truncated vectors will rank wrongly until you divide by the new length. The script re-normalizes in unit(v[:k]); your pipeline has to as well.
How do I use Matryoshka embeddings in practice?
Through the API. OpenAI’s embeddings guide says text-embedding-3-small returns 1,536 dimensions and text-embedding-3-large 3,072 by default, and that developers can shorten them, “remove some numbers from the end of the sequence”, without the embedding losing its concept-representing properties. The guide suggests passing the dimensions parameter when you create the embedding; if you cut vectors yourself, “you need to be sure to normalize” them. OpenAI does not describe its training method in that guide, so I describe this as Matryoshka-style behavior rather than claim a recipe.
With open models. Sentence Transformers can train with MatryoshkaLoss and load models with a truncate_dim setting, and the Hugging Face post uses nomic-ai/nomic-embed-text-v1.5 as an example of a released Matryoshka model. Check the model card: if it doesn’t say Matryoshka, truncation will behave like my “even” row.
In the database. pgvector can index a prefix with subvector(embedding, 1, 512) and then re-rank candidates by the full vector, per its README. That is the MRL paper’s adaptive retrieval idea in two SQL statements: a cheap shortlist on the short prefix, an exact ordering on the full vector.
What I would do
- Treat dimension count as a tuning knob, not a fixed property of the model. Embed at full size, store full size if you can afford the disk, and index a prefix.
- Measure on your own labelled queries. Run recall@10 at 1,536, 768, 512 and 256 dimensions and pick the smallest size that stays within a point or two of full size. My embedding model guide has a harness for this.
- Always re-normalize after truncating, and assert the norm in a test.
- Combine with lower precision. Truncation and quantization stack: a 512-dimension
halfvecis a sixth of the size of a 1,536-dimension float32 vector. - Keep the full vectors somewhere for re-ranking. The prefix finds candidates; the full vector decides the order.
FAQ
Why are they called Matryoshka embeddings?
After Matryoshka dolls, which nest inside one another. Each prefix of the vector is a smaller, complete embedding nested inside the larger one.
Can I truncate any embedding model?
You can, but quality drops fast unless the model was trained with a Matryoshka-style loss. Ordinary models spread information across all dimensions, so a prefix is just a partial signal.
Do I need to re-normalize after truncating?
Yes, if your similarity function assumes unit-length vectors. OpenAI’s guide says so explicitly for manual shortening, and the Hugging Face post recommends re-normalizing too.
Does OpenAI use Matryoshka embeddings?
OpenAI’s text-embedding-3 models support shortening through a dimensions parameter and by trimming numbers from the end. Their embeddings guide does not name the training method, so check OpenAI’s own documentation for that detail.
What is adaptive retrieval?
A two-stage search from the MRL paper: shortlist candidates using the first few dimensions, then re-rank the shortlist with more dimensions. It cuts the cost of the first stage without losing the precision of the full vector.
Sources
- Kusupati et al., Matryoshka Representation Learning, NeurIPS 2022 (arXiv May 2022, v4 February 2024)
- Aarsen, Xenova and Sanseviero, Introduction to Matryoshka Embedding Models, Hugging Face, February 23, 2024
- OpenAI, Embeddings guide, checked 2026-10-31
- pgvector README, version 0.8.7, checked 2026-10-31