MODEL-ADAPTIVE OPTIMIZATION FOR SPARSE INFERENCE
Mixture-of-Experts models waste most of their serving cost moving expert weights around. Strata profiles how your model routes, calibrates competing policies under an explicit quality budget, and deploys the measured winner.
RUN BIG MODELS ON THE LAPTOP YOU HAVE
Strata sizes each model's active working set to fit your memory budget, paging the rest of the experts from SSD — so a laptop can run a model far larger than its RAM. The working-set sizing is exact; the throughput below is an optimistic estimate until you verify it on your own device.
7 of 7 tested models fit a 16 GB working-memory budget · reserving ~4 GB for the OS
Won't fit in 16 GB resident (17.2 GB). Streamed — never assembles the full model — it loads and runs in a measured 12.2 GB peak (51% of experts hot, rest paged).
Won't fit in 16 GB resident (60.1 GB). Streamed — never assembles the full model — it loads and runs in a measured 11.3 GB peak (9% of experts hot, rest paged).
Loads fully in memory — runs natively, Strata paging isn't needed.
Won't fit in 16 GB resident (44.8 GB). Streamed — never assembles the full model — it loads and runs in a measured 11.8 GB peak (18% of experts hot, rest paged).
Won't fit in 16 GB resident (26.3 GB). Streamed — never assembles the full model — it loads and runs in a measured 10.5 GB peak (25% of experts hot, rest paged).
Won't fit in 16 GB resident (23.6 GB). Streamed — never assembles the full model — it loads and runs in a measured 10.2 GB peak (31% of experts hot, rest paged).
Loads fully in memory — runs natively, Strata paging isn't needed.
bit-exact = paged output is bit-identical to the resident model (deterministic). near-exact* = quality-neutral ~1-ULP drift, not bit-identical (Qwen-family, GLM). Fidelity is machine-independent — hover any chip for the full measured metrics.
Benchmark config: peaks measured end-to-end (load + generate) on Apple M5 Max, mlx-lm 0.31.3, 140-token generation, per-model expert budget, greedy · commit 6562877. Peak can vary with mlx-lm version, context length, allocator, and OS.
More classes landing next (paged-runtime support in progress): gpt-oss-20b
Real measurement: Qwen3-30B-A3B, single node, physical cache-bypassed reads, 6 prompts × 2 runs per point. The gap is the product.
Every number carries its quality cost. When nothing beats standard routing within your budget, Strata says so.
Trace which experts your model's router actually activates, token by token, on your workload.
Sweep competing routing policies in normalized units and find each one's safe operating point under your quality budget.
Race the calibrated policies on a real fixed-memory runtime with physical reads — the measured winner is the only one that counts.
One config file, one OpenAI-compatible endpoint. Your application code doesn't change.
Across eight MoE architectures we measured, the best routing policy flips with the model, the memory budget, and the quality tolerance — including the best published method, which Strata implements as a baseline and beats or adopts per deployment. A hard-coded choice is provably wrong somewhere. A measured tournament never is.