Malhar Ujawane

Notes

Note·

TIL: count a model's parameters with HTTP Range requests

A safetensors file starts with an 8-byte header length and a JSON header. Two Range requests per shard counted Mistral Large 4's 1,053B params from 6.2 MB.

You can get the exact parameter count of Mistral Large 4, whose 54 FP8 shards add up to 1,060.6 GB, without downloading any of it. A .safetensors file starts with an 8-byte little-endian integer, the length of a JSON header, and the header lists every tensor’s name, dtype, shape and byte offsets. Two HTTP Range requests per shard get you the header.

Here it is on the first shard, run on October 30, 2026 with curl 8.14.1:

U=https://huggingface.co/mistralai/Mistral-Large-4-1T-A52B/resolve/main/consolidated-00001-of-00054.safetensors

curl -sI -L "$U" | grep -i content-length | tail -1
# content-length: 19974069072

curl -sL -r 0-7 "$U" | od -An -tu8
#                112456

N=112456
curl -sL -r 8-$((8+N-1)) "$U" | python3 -c '
import json,sys
h=json.load(sys.stdin); h.pop("__metadata__",None)
print(len(h),"tensors")
k="layers.1.experts.0.w1.weight"; print(k,h[k]["dtype"],h[k]["shape"])
k="layers.1.experts.0.w1.qscale_weight"; print(k,h[k]["dtype"],h[k]["shape"])'
# 1038 tensors
# layers.1.experts.0.w1.weight F8_E4M3 [6144, 6144]
# layers.1.experts.0.w1.qscale_weight F32 [48, 48]

That’s 112,464 bytes read out of a 19,974,069,072-byte file. Multiply the shapes, sum across all 54 shards and you have the count: 1,053.16 billion parameters, from 6.2 MB of headers. The full Python script is in today’s Mistral Large 4 post.

Two details decide whether this works.

-L matters. Hugging Face answers /resolve/ with a 302 to its CDN. Without -L, curl hands od the redirect’s response body, and you get a page of meaningless 64-bit numbers instead of a header length:

curl -s -o /dev/null -w "%{http_code} %{redirect_url}\n" -r 0-7 "$U" | cut -c1-60
# 302 https://us.aws.cdn.hf.co/xet-bridge-us/6abd56613b342d55f

FP8 checkpoints carry scale tensors. Each FP8 weight block-scaled at 128 x 128 has a companion qscale_weight in F32, which is why the 6144 x 6144 expert weight above has a 48 x 48 scale. Those are quantization metadata, not parameters, so skip anything with qscale in its name before you sum.

This is the same move as my HN Algolia TIL and the offline OpenAI tokenizer: ask the file or the API for the small thing you need instead of the big thing it usually hands you.

Get new posts in your inbox

© 2026 Malhar Ujawane. Built with Astro.