AMD · Strix Halo · unified memory

What runs on a Ryzen AI Max+ 395 128GB

128 GB at 256 GB/s. The largest model that fits at Q4_K_M is gpt-oss 120B-A5.1B, at roughly 34.6 tokens per second. Unified memory means the operating system takes a share, so budget around three quarters of the total.

Runs comfortably

6.1 GB of 96.0 GB · 6%
096 GB
Weights 4.6 GB
KV cache 1.0 GB
Runtime overhead 0.5 GB

Llama 3.1 8B at Q4_K_M leaves 89.9 GB spare on a Ryzen AI Max+ 395 128GB. There is room to raise the context length or move up a quantisation level.

Generation26.2tok/s
Prompt processing1308tok/s
Max context128Ktokens
KV per 1K tokens0GB

Every quantisation of Llama 3.1 8B on a Ryzen AI Max+ 395 128GB

Highlighted row is the highest quality that still fits at 8K context.

QuantisationbpwWeights TotalVerdictMax ctxtok/s
FP16 / BF16 16.00 15.0 GB 16.5 GB Runs comfortably 128K 8.68
INT8 / W8A8 8.50 7.7 GB 9.2 GB Runs comfortably 128K 16.3
Q8_0 (GGUF) 8.50 7.7 GB 9.2 GB Runs comfortably 128K 16.3
FP8 (E4M3) 8.00 7.5 GB 9.0 GB Runs comfortably 128K 16.8
Q6_K 6.56 6.1 GB 7.7 GB Runs comfortably 128K 20.2
AWQ 4-bit 4.25 5.4 GB 6.9 GB Runs comfortably 128K 22.7
GPTQ 4-bit 4.25 5.4 GB 6.9 GB Runs comfortably 128K 22.7
MXFP4 4.25 5.4 GB 6.9 GB Runs comfortably 128K 22.7
Q5_K_M 5.67 5.3 GB 6.8 GB Runs comfortably 128K 23.1
Q5_K_S 5.52 5.2 GB 6.7 GB Runs comfortably 128K 23.6
Q4_K_M 4.85 4.6 GB 6.1 GB Runs comfortably 128K 26.2
Q4_K_S 4.58 4.4 GB 5.9 GB Runs comfortably 128K 27.3
Q4_0 4.55 4.4 GB 5.9 GB Runs comfortably 128K 27.5
IQ4_XS 4.25 4.1 GB 5.6 GB Runs comfortably 128K 28.9
Q3_K_M 3.91 3.8 GB 5.4 GB Runs comfortably 128K 30.7
IQ3_M 3.70 3.7 GB 5.2 GB Runs comfortably 128K 32.0
IQ3_XXS 3.06 3.2 GB 4.7 GB Runs comfortably 128K 36.5
Q2_K 2.63 2.8 GB 4.3 GB Runs comfortably 128K 40.3
IQ2_XXS 2.06 2.3 GB 3.9 GB Runs comfortably 128K 46.8
IQ1_M 1.75 2.1 GB 3.6 GB Runs comfortably 128K 51.3

Models on a Ryzen AI Max+ 395 128GB

Q4_K_M at 8K context. Bandwidth sets the speed; capacity sets the ceiling.

ModelParamsWeightsVerdictMax ctxtok/s
DeepSeek-R1 671B-A37B 671B 379.0 GB Won't fit — 1.90
Qwen3 235B-A22B 235.1B 132.8 GB Won't fit — 4.85
gpt-oss 120B-A5.1B 116.8B 66.0 GB Runs comfortably 128K 34.6
Llama 4 Scout 109B-A17B 109B 61.7 GB Runs comfortably 180K 11.1
Qwen2.5 72B 72.7B 41.2 GB Runs comfortably 128K 3.16
Llama 3.1 70B 70.5B 40.0 GB Runs comfortably 128K 3.26
Mixtral 8x7B 46.7B 26.4 GB Runs comfortably 32K 14.7
Command R 35B 35.0B 20.1 GB Runs comfortably 60K 5.36
Yi-1.5 34B 34.4B 19.5 GB Runs comfortably 32K 6.57
Qwen3 32B 32.8B 18.6 GB Runs comfortably 128K 6.84
Qwen2.5-Coder 32B 32.8B 18.6 GB Runs comfortably 128K 6.84
DeepSeek-R1-Distill-Qwen 32B 32.8B 18.6 GB Runs comfortably 128K 6.84
Qwen3 30B-A3B 30.5B 17.3 GB Runs comfortably 128K 42.2
Gemma 3 27B 27.4B 15.6 GB Runs comfortably 128K 8.21
Mistral Small 24B 23.6B 13.4 GB Runs comfortably 32K 9.56
gpt-oss 20B-A3.6B 20.9B 11.9 GB Runs comfortably 128K 49.1
Qwen3 14B 14.8B 8.5 GB Runs comfortably 128K 14.8
Phi-4 14B 14.7B 8.4 GB Runs comfortably 16K 14.6
Mistral NeMo 12B 12.3B 7.0 GB Runs comfortably 128K 17.5
Gemma 3 12B 12.2B 7.0 GB Runs comfortably 128K 17.8
GLM-4 9B 9.4B 5.4 GB Runs comfortably 128K 24.1
Qwen3 8B 8.2B 4.7 GB Runs comfortably 128K 25.3
Llama 3.1 8B 8.0B 4.6 GB Runs comfortably 128K 26.2
DeepSeek-R1-Distill-Llama 8B 8.0B 4.6 GB Runs comfortably 128K 26.2
Qwen2.5 7B 7.6B 4.4 GB Runs comfortably 128K 29.1
Mistral 7B v0.3 7.3B 4.1 GB Runs comfortably 32K 29.0
Gemma 3 4B 4.3B 2.5 GB Runs comfortably 128K 49.6
Qwen3 4B 4.0B 2.3 GB Runs comfortably 128K 46.5
Llama 3.2 3B 3.2B 1.8 GB Runs comfortably 128K 58.4
Llama 3.2 1B 1.2B 0.7 GB Runs comfortably 128K 157

Direct answers

Llama 3.1 8B on Ryzen AI Max+ 395 128GB
8.0BLlama 3.1 70B on Ryzen AI Max+ 395 128GB
70.5BLlama 3.2 3B on Ryzen AI Max+ 395 128GB
3.2BLlama 3.2 1B on Ryzen AI Max+ 395 128GB
1.2BLlama 4 Scout 109B-A17B on Ryzen AI Max+ 395 128GB
109BQwen3 8B on Ryzen AI Max+ 395 128GB
8.2BQwen3 14B on Ryzen AI Max+ 395 128GB
14.8BQwen3 32B on Ryzen AI Max+ 395 128GB
32.8B