AMD · RDNA3

What runs on a Radeon RX 7900 XTX

24 GB at 960 GB/s. The largest model that fits at Q4_K_M is Qwen3 32B, at roughly 27.6 tokens per second.

Model

Hardware

Quantisation

Context 8K

KV cache precision

Runtime

Batch

GPUs

Monitor plugged into this GPU

Runs comfortably

6.4 GB of 21.8 GB · 30%

022 GB

Weights 4.6 GB

KV cache 1.0 GB

Runtime overhead 0.8 GB

Llama 3.1 8B at Q4_K_M leaves 15.3 GB spare on a Radeon RX 7900 XTX. There is room to raise the context length or move up a quantisation level.

Generation105tok/s

Prompt processing3217tok/s

Max context128Ktokens

KV per 1K tokens0GB

Every quantisation of Llama 3.1 8B on a Radeon RX 7900 XTX

Highlighted row is the highest quality that still fits at 8K context.

Quantisation	bpw	Weights	Total	Verdict	Max ctx	tok/s
FP16 / BF16	16.00	15.0 GB	16.8 GB	Runs comfortably	48K	35.1
INT8 / W8A8	8.50	7.7 GB	9.5 GB	Runs comfortably	106K	65.7
Q8_0 (GGUF)	8.50	7.7 GB	9.5 GB	Runs comfortably	106K	65.7
FP8 (E4M3)	8.00	7.5 GB	9.3 GB	Runs comfortably	108K	67.6
Q6_K	6.56	6.1 GB	8.0 GB	Runs comfortably	118K	81.1
AWQ 4-bit	4.25	5.4 GB	7.2 GB	Runs comfortably	124K	90.8
GPTQ 4-bit	4.25	5.4 GB	7.2 GB	Runs comfortably	124K	90.8
MXFP4	4.25	5.4 GB	7.2 GB	Runs comfortably	124K	90.8
Q5_K_M	5.67	5.3 GB	7.1 GB	Runs comfortably	125K	92.5
Q5_K_S	5.52	5.2 GB	7.0 GB	Runs comfortably	126K	94.8
Q4_K_M	4.85	4.6 GB	6.4 GB	Runs comfortably	128K	105
Q4_K_S	4.58	4.4 GB	6.2 GB	Runs comfortably	128K	109
Q4_0	4.55	4.4 GB	6.2 GB	Runs comfortably	128K	110
IQ4_XS	4.25	4.1 GB	5.9 GB	Runs comfortably	128K	115
Q3_K_M	3.91	3.8 GB	5.7 GB	Runs comfortably	128K	123
IQ3_M	3.70	3.7 GB	5.5 GB	Runs comfortably	128K	128
IQ3_XXS	3.06	3.2 GB	5.0 GB	Runs comfortably	128K	145
Q2_K	2.63	2.8 GB	4.6 GB	Runs comfortably	128K	160
IQ2_XXS	2.06	2.3 GB	4.2 GB	Runs comfortably	128K	185
IQ1_M	1.75	2.1 GB	3.9 GB	Runs comfortably	128K	202

Q4_K_M at 8K context. Bandwidth sets the speed; capacity sets the ceiling.

Model	Params	Weights	Verdict	Max ctx	tok/s
DeepSeek-R1 671B-A37B	671B	379.0 GB	Won't fit	—	1.78
Qwen3 235B-A22B	235.1B	132.8 GB	Won't fit	—	3.07
gpt-oss 120B-A5.1B	116.8B	66.0 GB	Won't fit	—	15.6
Llama 4 Scout 109B-A17B	109B	61.7 GB	Won't fit	—	4.91
Qwen2.5 72B	72.7B	41.2 GB	Won't fit	—	1.44
Llama 3.1 70B	70.5B	40.0 GB	Won't fit	—	1.52
Mixtral 8x7B	46.7B	26.4 GB	Won't fit	—	15.0
Command R 35B	35.0B	20.1 GB	Won't fit	640	2.84
Yi-1.5 34B	34.4B	19.5 GB	Won't fit	6K	18.1
Qwen3 32B	32.8B	18.6 GB	Fits, but tight	9K	27.6
Qwen2.5-Coder 32B	32.8B	18.6 GB	Fits, but tight	9K	27.6
DeepSeek-R1-Distill-Qwen 32B	32.8B	18.6 GB	Fits, but tight	9K	27.6
Qwen3 30B-A3B	30.5B	17.3 GB	Runs comfortably	39K	112
Gemma 3 27B	27.4B	15.6 GB	Runs comfortably	57K	33.1
Mistral Small 24B	23.6B	13.4 GB	Runs comfortably	32K	38.6
gpt-oss 20B-A3.6B	20.9B	11.9 GB	Runs comfortably	128K	173
Qwen3 14B	14.8B	8.5 GB	Runs comfortably	80K	59.3
Phi-4 14B	14.7B	8.4 GB	Runs comfortably	16K	58.8
Mistral NeMo 12B	12.3B	7.0 GB	Runs comfortably	89K	70.3
Gemma 3 12B	12.2B	7.0 GB	Runs comfortably	128K	71.1
GLM-4 9B	9.4B	5.4 GB	Runs comfortably	128K	96.0
Qwen3 8B	8.2B	4.7 GB	Runs comfortably	115K	101
Llama 3.1 8B	8.0B	4.6 GB	Runs comfortably	128K	105
DeepSeek-R1-Distill-Llama 8B	8.0B	4.6 GB	Runs comfortably	128K	105
Qwen2.5 7B	7.6B	4.4 GB	Runs comfortably	128K	116
Mistral 7B v0.3	7.3B	4.1 GB	Runs comfortably	32K	116
Gemma 3 4B	4.3B	2.5 GB	Runs comfortably	128K	196
Qwen3 4B	4.0B	2.3 GB	Runs comfortably	128K	183
Llama 3.2 3B	3.2B	1.8 GB	Runs comfortably	128K	230
Llama 3.2 1B	1.2B	0.7 GB	Runs comfortably	128K	609