GLM-4.5-Air-4bit
quality budget ≤ +0.5% perplexity · research dataset
+76.4%
throughput, same memory
11.71
tok/s at budget 48 (21 GB)
cache @ 0.498σ
selected policy · quality -0.39%
Policy tournament
Every configuration measured on the real paging runtime with physical (cache-bypassed) expert reads. Residency-bias policy per arXiv:2412.00099.
| budget | resident | policy | bias | tok/s | p95 | reads/tok | quality | |
|---|---|---|---|---|---|---|---|---|
| 48 | 21 GB | cache | 0.996σ | 16.57 | 94.8 ms | 192.8 MB | +2.09% | |
| 48 | 21 GB | warm | 0.996σ | 14.97 | 110.2 ms | 211.3 MB | +1.57% | |
| 48 | 21 GB | cache | 0.498σ | 11.71 | 132.8 ms | 347 MB | -0.39% | winner |
| 48 | 21 GB | warm | 0.498σ | 11.48 | 139 ms | 383.6 MB | -0.42% | |
| 48 | 21 GB | none | — | 6.64 | 227.5 ms | 777 MB | 0% | |
| 64 | 28 GB | warm | 0.996σ | 21.66 | 69.2 ms | 117.8 MB | +1.57% | |
| 64 | 28 GB | cache | 0.996σ | 17.15 | 87.2 ms | 108.1 MB | +2.09% | |
| 64 | 28 GB | warm | 0.498σ | 15.95 | 100 ms | 222.3 MB | -0.42% | |
| 64 | 28 GB | cache | 0.498σ | 13.99 | 117 ms | 187.4 MB | -0.39% | |
| 64 | 28 GB | none | — | 9.67 | 154.7 ms | 476.2 MB | 0% |
Provenance: model mlx-community/GLM-4.5-Air-4bit · quality budget +0.5% · single-node MLX paging runtime. Rows flagged degenerate produced looping output and are excluded from selection.