A query encoder as a 120 MiB file.
In the browser. Open source.

SciFact test split, nDCG@10 against the model's own fp32 index (0.7559): 120 MiB at 2.1 bits per weight 98.2 %, 105 MiB at 1.8 bits 96.5 %, 91 MiB at 1.6 bits 93.6 % (torch simulation of the container, synthetic-query calibration, 4-d codebooks per 256-column block, input-side structured rotation). The browser reproduces the simulation: 0.7375 vs 0.7380 on the 300 test queries, per-query cosine to the reference 1.00000. The released SciDocs containers of the same recipe (127 / 113 MiB) hold 94.7 / 91.3 % and were verified in the browser to ±0.001; the SciFact files are on Hugging Face as well. Below ~180 MiB llama.cpp has no format at all; BitNet-270m gets to 140 MiB only by training the model ternary.

Idle laptop, Intel integrated GPU, headless Chrome, 300 SciFact queries: our runtime p50 103 ms / p95 113 ms, load 2.4 s, peak process memory 1 592 MiB; llama.cpp in wllama (WebGPU) p50 444 ms / p95 754 ms, load 4.2 s, 2 080 MiB, at +0.011 nDCG@10 for the larger file. The runtime decodes the codebook weights once per query, runs the 28 blocks of the encoder in 17 dispatches per block and needs no WebAssembly. One device, one model, one corpus; discrete GPUs and phones are not measured yet. Runtime: client/vqweb in the repository, Apache-2.0.

bge-m3 Q4_K_M with a 4-bit token table: cosine to the fp32 query vector 0.990, top-10 overlap 0.864; the 321 MiB Q3_K file keeps 98.3 %. Qwen3-Embedding-0.6B: 98.7 % at 340 MiB, 99.5 % at 385 MiB. The same Czech-calibrated files keep 99.6–100 % on SciFact. What made it work: at low bit widths the language of the calibration text matters more than its domain (+0.09 nDCG@10 for Czech over English text at 2.6 bits, domain another +0.01–0.02), and tolerance to compression is a property of the checkpoint (one 0.6B model holds 99 % at 3.4 bits, another of the same architecture fails below 4.5). The vector-quantised Czech client (released 2026-09-16, thinletter/qwen3-embedding-0.6b-vq-clients): two-dimensional codebooks with 128 entries at 3.57 bits per weight plus the full token table, 237 MiB, 96.8 % of fp32 nDCG@10 on WebFAQ-cs (71 529 passages, 7 231 human-judged questions), cosine to the fp32 query vector 0.946, top-10 overlap 0.755; the browser reproduces the simulation on 50 queries to the fourth decimal at 112 ms per query. Two-dimensional codebooks beat four-dimensional ones per bit; the token table costs 2 points against the 98.8 % of the blocks alone, so a 4-bit table is the next format step. One seed; the Czech legal index gives 93.1 % (saturated synthetic test, calibration from another domain).

The prompt in front of every query is the same 19 tokens, so the full-precision model computes its keys and values once and the compressed model only reads the user's words. Calibrating every block with the exact prompt in place gives +0.013 / +0.009 / +0.008 nDCG@10 over the best query-calibrated client of the same seed at 1.8 bits on SciFact (cosine to fp32 0.90 → 0.92) and +0.007 / +0.016 / +0.010 on the Czech legal index; nothing at 2.1 or 1.6 bits and nothing on SciDocs or NFCorpus, so the released files stay as they are and the recipe is an option in the runtime. What the two winning corpora share: calibration queries shaped like the real ones. Pre-registered 2026-09-10/11, scored against the prediction, published with the negative arms.

Repository: https://github.com/rosecky/embedding-quantization-public. It carries the data layer and metrics, the recipe to verify any client against your own index (import, synthetic queries, quantise, paired bootstrap), the GPTQ exporter onto the llama.cpp K-quant grids and the llama-quantize recipes of the released files, the vector-quantisation quantiser and compiler, the torch reference, the WebGPU runtime with its verification ladder (bit-exact dequantisation, per-op activations, end-to-end browser check), and the demo. Method: every run is pre-registered with a prediction and a kill rule, comparisons change one thing at a time, differences are paired bootstrap intervals over queries (10 000 draws) read against calibration-draw variance (~0.01) and between-machine variation (±0.006). The report states which claims held, which failed and which are ties.

SciFact 98.9 %, NFCorpus 99.3 %, ArguAna 101 %, SciDocs 99.2 % (microsoft/harrier-oss-v1-0.6b, GPTQ on the llama.cpp Q3_K grid, generic English calibration). At this bit width llama.cpp's own quantiser reaches the same quality with the same text; at ~200 MiB our export is not a better file, and the report says so. The value is the verified pipeline and the calibration map, not the file.

The idea: give the neurons the corpus uses most a larger codebook and the rest a smaller one at the same average rate. Three rotation seeds, matched random control: importance minus random −0.0085 cosine to fp32, nDCG@10 within noise. Side finding: two quantisations that differ only in which rows share a codebook differ by up to 0.03 cosine, so on this grid an effect under 0.01 needs three seeds or a paired construction. Pre-registered and closed 2026-09-10.

Qwen3-Embedding-0.6B is an order of magnitude more fragile under compression than harrier or jina of the same architecture: every file at or below 3 bits fails on Czech, the released client is a 4.9-bit K-quant (340 MiB). The 2.1-bit vector-quantised point gave 65.6 % of fp32 on the laptop (torch 2.6, cu124) and 80.6 % on a rented box (torch 2.11, cu128) from the identical configuration; the 3.1-bit point 96.3 % vs 96.5 %. Read numbers below ~85 % for this model as a regime, not a value. Measured 2026-09-15/16 on the public WebFAQ-cs benchmark.