Legal CS over the full production index (Elasticsearch, side by side with the original fp32 model) → legal.html
checking WebGPU …
The file is fetched as 24 MiB chunks from this site and kept in the browser's private storage (OPFS), so the next visit starts without downloading. Nothing is sent anywhere.
On this device
- Model file
- …
- Download
- –
- Load (wasm + weights)
- –
- Backend
- –
- Threads
- –
- Cross-origin isolated
- –
- Memory after load
- –
- Encode, last / p50
- –
- Search … docs, last / p50
- –
- Queries this session
- 0
Reference, same file, native llama.cpp on a laptop (8 threads, ctx 512, idle): 151 ms per query, 928 MiB peak RSS. Latency in a busy tab or on a loaded machine is higher.
Try a test query:
Verify on this device
Runs the … … test queries through the model in this tab, one by one, scores each against the index and computes nDCG@10 with the same code path as the search box. Takes … × (your latency); abortable.
| nDCG@10 | vs native | Latency p50 | p95 | |
|---|---|---|---|---|
| This device (browser, this run) | – | – | – | – |
| Native llama.cpp, same file | 0.7440 | – | 151 ms | – |
| Browser, our measurement | 0.7425 | – | under load, see README | |
| Original fp16 model (1 143 MiB) | 0.7559 | – | not in the browser | |
Vector-quantised clients: results
Below llama.cpp's smallest format (IQ2_XS, 177.5 MiB) we run a vector-quantised container (4-d codebooks, 1.8–2.1 bits per weight) with its own WebGPU runtime. The runtime is not public yet; the numbers are. Same laptop, integrated Intel GPU, headless Chrome, 300 SciFact test queries, machine idle.
| runtime / file | MiB | load | peak RSS | p50 | p95 | nDCG@10 |
|---|---|---|---|---|---|---|
| VQ runtime, 2.10 bpw | 119.5 | 2.4 s | 1 592 MiB | 103 ms | 113 ms | 0.7375 |
| VQ runtime, 1.83 bpw | 105.3 | 2.5 s | 1 501 MiB | 102 ms | 112 ms | 0.7109 |
| wllama WebGPU, Q3_K (this page) | 235.1 | 4.2 s | 2 080 MiB | 444 ms | 754 ms | 0.7481 |
| wllama WebGPU, IQ2_M | 199.3 | 5.2 s | 1 983 MiB | 508 ms | 1 012 ms | 0.7387 |
| wllama WASM 8 threads, Q2_K | 192.4 | 4.4 s | 1 228 MiB | 1 669 ms | 3 363 ms | 0.7425 |
fp32 model: 0.7559. On the Czech legal index the same format at 2.1 bpw keeps only 83 % (jina) / 64 % (Qwen3-Embedding) of fp32, so a ~3-bit variant is needed there. Full tables, caveats and the paired comparison with BitNet-270m: vq_results.md. A live demo with the VQ clients runs behind access control; ask for an invitation (contact in the repository README).