GLM-4.5-Air-4bit

quality budget ≤ +0.5% perplexity · research dataset

+76.4%
throughput, same memory
11.71
tok/s at budget 48 (21 GB)
cache @ 0.498σ
selected policy · quality -0.39%
standard6.64warm @ 0.498σ11.48cache @ 0.498σ11.71warm @ 0.996σ14.97cache @ 0.996σ16.57tok/s at budget 48 · error bars are 95% CI where measured

Policy tournament

Every configuration measured on the real paging runtime with physical (cache-bypassed) expert reads. Residency-bias policy per arXiv:2412.00099.

budgetresidentpolicybiastok/sp95reads/tokquality
4821 GBcache0.996σ16.5794.8 ms192.8 MB+2.09%
4821 GBwarm0.996σ14.97110.2 ms211.3 MB+1.57%
4821 GBcache0.498σ11.71132.8 ms347 MB-0.39%winner
4821 GBwarm0.498σ11.48139 ms383.6 MB-0.42%
4821 GBnone—6.64227.5 ms777 MB0%
6428 GBwarm0.996σ21.6669.2 ms117.8 MB+1.57%
6428 GBcache0.996σ17.1587.2 ms108.1 MB+2.09%
6428 GBwarm0.498σ15.95100 ms222.3 MB-0.42%
6428 GBcache0.498σ13.99117 ms187.4 MB-0.39%
6428 GBnone—9.67154.7 ms476.2 MB0%

Provenance: model mlx-community/GLM-4.5-Air-4bit · quality budget +0.5% · single-node MLX paging runtime. Rows flagged degenerate produced looping output and are excluded from selection.