Apple · M2 · unified memory

What runs on a Mac Studio M2 Max 96GB

96 GB at 400 GB/s. The largest model that fits at Q4_K_M is gpt-oss 120B-A5.1B, at roughly 56.5 tokens per second. Unified memory means the operating system takes a share, so budget around three quarters of the total.

Runs comfortably

6.1 GB of 72.0 GB · 9%
072 GB
Weights 4.6 GB
KV cache 1.0 GB
Runtime overhead 0.5 GB

Llama 3.1 8B at Q4_K_M leaves 65.9 GB spare on a Mac Studio M2 Max 96GB. There is room to raise the context length or move up a quantisation level.

Generation44.1tok/s
Prompt processing706tok/s
Max context128Ktokens
KV per 1K tokens0GB

Every quantisation of Llama 3.1 8B on a Mac Studio M2 Max 96GB

Highlighted row is the highest quality that still fits at 8K context.

QuantisationbpwWeights TotalVerdictMax ctxtok/s
FP16 / BF16 16.00 15.0 GB 16.5 GB Runs comfortably 128K 14.7
INT8 / W8A8 8.50 7.7 GB 9.2 GB Runs comfortably 128K 27.6
Q8_0 (GGUF) 8.50 7.7 GB 9.2 GB Runs comfortably 128K 27.6
FP8 (E4M3) 8.00 7.5 GB 9.0 GB Runs comfortably 128K 28.4
Q6_K 6.56 6.1 GB 7.7 GB Runs comfortably 128K 34.1
AWQ 4-bit 4.25 5.4 GB 6.9 GB Runs comfortably 128K 38.2
GPTQ 4-bit 4.25 5.4 GB 6.9 GB Runs comfortably 128K 38.2
MXFP4 4.25 5.4 GB 6.9 GB Runs comfortably 128K 38.2
Q5_K_M 5.67 5.3 GB 6.8 GB Runs comfortably 128K 38.9
Q5_K_S 5.52 5.2 GB 6.7 GB Runs comfortably 128K 39.9
Q4_K_M 4.85 4.6 GB 6.1 GB Runs comfortably 128K 44.1
Q4_K_S 4.58 4.4 GB 5.9 GB Runs comfortably 128K 46.1
Q4_0 4.55 4.4 GB 5.9 GB Runs comfortably 128K 46.3
IQ4_XS 4.25 4.1 GB 5.6 GB Runs comfortably 128K 48.7
Q3_K_M 3.91 3.8 GB 5.4 GB Runs comfortably 128K 51.8
IQ3_M 3.70 3.7 GB 5.2 GB Runs comfortably 128K 53.9
IQ3_XXS 3.06 3.2 GB 4.7 GB Runs comfortably 128K 61.5
Q2_K 2.63 2.8 GB 4.3 GB Runs comfortably 128K 67.9
IQ2_XXS 2.06 2.3 GB 3.9 GB Runs comfortably 128K 78.8
IQ1_M 1.75 2.1 GB 3.6 GB Runs comfortably 128K 86.3

Models on a Mac Studio M2 Max 96GB

Q4_K_M at 8K context. Bandwidth sets the speed; capacity sets the ceiling.

ModelParamsWeightsVerdictMax ctxtok/s
DeepSeek-R1 671B-A37B 671B 379.0 GB Won't fit — 1.99
Qwen3 235B-A22B 235.1B 132.8 GB Won't fit — 4.59
gpt-oss 120B-A5.1B 116.8B 66.0 GB Fits, but tight 78K 56.5
Llama 4 Scout 109B-A17B 109B 61.7 GB Runs comfortably 52K 18.7
Qwen2.5 72B 72.7B 41.2 GB Runs comfortably 97K 5.34
Llama 3.1 70B 70.5B 40.0 GB Runs comfortably 101K 5.50
Mixtral 8x7B 46.7B 26.4 GB Runs comfortably 32K 24.7
Command R 35B 35.0B 20.1 GB Runs comfortably 41K 9.06
Yi-1.5 34B 34.4B 19.5 GB Runs comfortably 32K 11.1
Qwen3 32B 32.8B 18.6 GB Runs comfortably 128K 11.6
Qwen2.5-Coder 32B 32.8B 18.6 GB Runs comfortably 128K 11.6
DeepSeek-R1-Distill-Qwen 32B 32.8B 18.6 GB Runs comfortably 128K 11.6
Qwen3 30B-A3B 30.5B 17.3 GB Runs comfortably 128K 63.9
Gemma 3 27B 27.4B 15.6 GB Runs comfortably 128K 13.9
Mistral Small 24B 23.6B 13.4 GB Runs comfortably 32K 16.2
gpt-oss 20B-A3.6B 20.9B 11.9 GB Runs comfortably 128K 80.4
Qwen3 14B 14.8B 8.5 GB Runs comfortably 128K 24.9
Phi-4 14B 14.7B 8.4 GB Runs comfortably 16K 24.7
Mistral NeMo 12B 12.3B 7.0 GB Runs comfortably 128K 29.6
Gemma 3 12B 12.2B 7.0 GB Runs comfortably 128K 30.0
GLM-4 9B 9.4B 5.4 GB Runs comfortably 128K 40.6
Qwen3 8B 8.2B 4.7 GB Runs comfortably 128K 42.7
Llama 3.1 8B 8.0B 4.6 GB Runs comfortably 128K 44.1
DeepSeek-R1-Distill-Llama 8B 8.0B 4.6 GB Runs comfortably 128K 44.1
Qwen2.5 7B 7.6B 4.4 GB Runs comfortably 128K 49.0
Mistral 7B v0.3 7.3B 4.1 GB Runs comfortably 32K 48.9
Gemma 3 4B 4.3B 2.5 GB Runs comfortably 128K 83.4
Qwen3 4B 4.0B 2.3 GB Runs comfortably 128K 78.2
Llama 3.2 3B 3.2B 1.8 GB Runs comfortably 128K 98.2
Llama 3.2 1B 1.2B 0.7 GB Runs comfortably 128K 263

Direct answers

Llama 3.1 8B on Mac Studio M2 Max 96GB
8.0BLlama 3.1 70B on Mac Studio M2 Max 96GB
70.5BLlama 3.2 3B on Mac Studio M2 Max 96GB
3.2BLlama 3.2 1B on Mac Studio M2 Max 96GB
1.2BLlama 4 Scout 109B-A17B on Mac Studio M2 Max 96GB
109BQwen3 8B on Mac Studio M2 Max 96GB
8.2BQwen3 14B on Mac Studio M2 Max 96GB
14.8BQwen3 32B on Mac Studio M2 Max 96GB
32.8B