Qwen · 14.8B parameters

Qwen3 14B VRAM requirements

Qwen3 14B has 40 layers and uses grouped-query attention (8 KV heads). At Q4_K_M the weights come to 8.5 GB, and the best quantisation that fits a 24 GB card is INT8 / W8A8.

Model

Hardware

Quantisation

Context 8K

KV cache precision

Runtime

Batch

GPUs

Monitor plugged into this GPU

Runs comfortably

10.5 GB of 21.8 GB · 48%

022 GB

Weights 8.5 GB

KV cache 1.3 GB

Runtime overhead 0.8 GB

Qwen3 14B at Q4_K_M leaves 11.2 GB spare on a RTX 4090. There is room to raise the context length or move up a quantisation level.

Generation65.5tok/s

Prompt processing2346tok/s

Max context80Ktokens

KV per 1K tokens0GB

Every quantisation of Qwen3 14B on a RTX 4090

Highlighted row is the highest quality that still fits at 8K context.

Quantisation	bpw	Weights	Total	Verdict	Max ctx	tok/s
FP16 / BF16	16.00	27.5 GB	29.6 GB	Won't fit	—	3.95
INT8 / W8A8	8.50	14.3 GB	16.4 GB	Runs comfortably	43K	40.2
Q8_0 (GGUF)	8.50	14.3 GB	16.4 GB	Runs comfortably	43K	40.2
FP8 (E4M3)	8.00	13.8 GB	15.8 GB	Runs comfortably	46K	41.6
Q6_K	6.56	11.3 GB	13.4 GB	Runs comfortably	62K	50.2
Q5_K_M	5.67	9.7 GB	11.8 GB	Runs comfortably	72K	57.5
Q5_K_S	5.52	9.5 GB	11.6 GB	Runs comfortably	73K	58.9
AWQ 4-bit	4.25	9.4 GB	11.5 GB	Runs comfortably	74K	59.2
GPTQ 4-bit	4.25	9.4 GB	11.5 GB	Runs comfortably	74K	59.2
MXFP4	4.25	9.4 GB	11.5 GB	Runs comfortably	74K	59.2
Q4_K_M	4.85	8.5 GB	10.5 GB	Runs comfortably	80K	65.5
Q4_K_S	4.58	8.0 GB	10.1 GB	Runs comfortably	82K	68.6
Q4_0	4.55	8.0 GB	10.1 GB	Runs comfortably	83K	69.0
IQ4_XS	4.25	7.5 GB	9.6 GB	Runs comfortably	86K	72.8
Q3_K_M	3.91	7.0 GB	9.1 GB	Runs comfortably	89K	77.7
IQ3_M	3.70	6.7 GB	8.8 GB	Runs comfortably	91K	81.0
IQ3_XXS	3.06	5.7 GB	7.8 GB	Runs comfortably	97K	93.4
Q2_K	2.63	5.0 GB	7.1 GB	Runs comfortably	102K	104
IQ2_XXS	2.06	4.2 GB	6.3 GB	Runs comfortably	107K	122
IQ1_M	1.75	3.7 GB	5.8 GB	Runs comfortably	110K	136

Qwen3 14B on each GPU

Q4_K_M weights at 8K context, single card, monitor attached.

GPU	VRAM	GB/s	Verdict	Max ctx	tok/s
H100 SXM 80GB	80	3350	Runs comfortably	128K	243
A100 80GB	80	2039	Runs comfortably	128K	136
RTX 5090	32	1792	Runs comfortably	128K	126
RTX 5080	16	960	Runs comfortably	32K	68.7
RTX 4090	24	1008	Runs comfortably	80K	65.5
RTX 5070 Ti	16	896	Runs comfortably	32K	64.2
RTX 3090	24	936	Runs comfortably	80K	63.5
Radeon RX 7900 XTX	24	960	Runs comfortably	80K	59.3
L40S	48	864	Runs comfortably	128K	56.3
RTX A6000	48	768	Runs comfortably	128K	52.3
Mac Studio M3 Ultra 256GB	256	819	Runs comfortably	128K	51.5
RTX 4080 Super	16	736	Runs comfortably	32K	48.0
RTX 4070 Ti Super	16	672	Runs comfortably	32K	43.9
RTX 5070	12	672	Won't fit	8K	39.1
Mac Studio M4 Max 128GB	128	546	Runs comfortably	128K	38.3
RTX 5060 Ti 16GB	16	448	Runs comfortably	32K	32.4
RTX 4070 Super	12	504	Won't fit	8K	28.2
RTX 4070	12	504	Won't fit	8K	28.2
RTX 3060 12GB	12	360	Won't fit	8K	22.1
Arc B580	12	456	Won't fit	8K	22.0
NVIDIA DGX Spark (GB10)	128	273	Runs comfortably	128K	19.8
Mac Mini M4 Pro 48GB	48	273	Runs comfortably	128K	19.3
RTX 4060 Ti 16GB	16	288	Runs comfortably	32K	18.9
Ryzen AI Max+ 395 128GB	128	256	Runs comfortably	128K	14.8
RTX 3080 10GB	10	760	Won't fit	—	14.0

Architecture

Parameters	14.8B
Layers	40
Hidden size	5120
Attention heads / KV heads	40 / 8
Head dimension	128
Vocabulary	151,936
Trained context	128K
KV cache per 1K tokens	0 GB
Hugging Face	Qwen/Qwen3-14B

Direct answers

Qwen3 14B on RTX 5090
See the verdict Qwen3 14B on RTX 4090
See the verdict Qwen3 14B on RTX 3090
See the verdict Qwen3 14B on RTX 5080
See the verdict Qwen3 14B on RTX 5070 Ti
See the verdict Qwen3 14B on RTX 5070
See the verdict Qwen3 14B on RTX 4070 Ti Super
See the verdict Qwen3 14B on RTX 4070 Super
See the verdict Qwen3 14B on RTX 4070
See the verdict