Pick a model, pick a chip. The table shows every quant, whether it fits,
and how much context you get.
Model
Chip
KV cache
Quant
Weights
KV
Total
Left
~tok/s
Fits
01Weight sizes
Real .gguf sizes from HuggingFace where available.
Others are params × bits / 8, shown in italic.
02GiB vs GB
All sizes here are in GiB (1024³ bytes). HuggingFace lists
sizes in GB (1000³ bytes). A file HF calls 16.46 GB is 15.33 GiB.
Both are correct, different units. Multiply GiB by 1.074 to get GB.
03KV cache
Only full-attention layers count. Qwen3.8-27B has 16 of 64.
DeepSeek V4 uses MLA with a smaller footprint than GQA.
04Memory budget
macOS ~8 GiB + runtime ~4 GiB = 12 GiB reserved.
500 GiB usable on the Ultra 512.
05Speed estimate
Bandwidth ceiling from bandwidth / bytes_read_per_token * 0.75.
Ultra: 1,200 GB/s (both 64 and 80-core). Max 40-core: 614 GB/s. Max 32-core: 460 GB/s.
For MoE models, only active parameters are read per token. For n-gram models (Flash-Next),
the ~50 GiB embedding table is a sparse gather, not a sequential read, so it is excluded
from the per-token bandwidth calc.
* Fine-grained MoE caveat: models with many small experts (e.g. 512 experts at dim 640)
have significant expert-gather overhead. Real speed on Apple Silicon lands well below
the bandwidth ceiling shown here. Treat these numbers as an upper bound, not a prediction.