How much VRAM does Phi-4 14B need?

At Q4_K_M the weights are 8.4 GB. With an 8K context the KV cache adds 1.6 GB and the runtime adds 0.8 GB, for 10.8 GB in total.

Microsoft on NVIDIA Ada

Can I run Phi-4 14B on an L40S?

Yes. Phi-4 14B at Q4_K_M uses 10.8 GB of the 44.3 GB available on a L40S, and runs at about 55.8 tokens per second. You can push the context to 16K.

Model

Hardware

Quantisation

Context 8K

KV cache precision

Runtime

Batch

GPUs

Monitor plugged into this GPU

Runs comfortably

10.8 GB of 44.3 GB · 24%

044 GB

Weights 8.4 GB

KV cache 1.6 GB

Runtime overhead 0.8 GB

Phi-4 14B at Q4_K_M leaves 33.5 GB spare on a L40S. There is room to raise the context length or move up a quantisation level.

Generation55.8tok/s

Prompt processing2586tok/s

Max context16Ktokens

KV per 1K tokens0GB

Highlighted row is the highest quality that still fits at 8K context.

Quantisation	bpw	Weights	Total	Verdict	Max ctx	tok/s
FP16 / BF16	16.00	27.4 GB	29.8 GB	Runs comfortably	16K	18.3
INT8 / W8A8	8.50	14.3 GB	16.7 GB	Runs comfortably	16K	34.0
Q8_0 (GGUF)	8.50	14.3 GB	16.7 GB	Runs comfortably	16K	34.0
FP8 (E4M3)	8.00	13.7 GB	16.1 GB	Runs comfortably	16K	35.5
Q6_K	6.56	11.2 GB	13.6 GB	Runs comfortably	16K	42.7
Q5_K_M	5.67	9.7 GB	12.1 GB	Runs comfortably	16K	48.8
Q5_K_S	5.52	9.4 GB	11.8 GB	Runs comfortably	16K	50.1
AWQ 4-bit	4.25	8.7 GB	11.1 GB	Runs comfortably	16K	54.1
GPTQ 4-bit	4.25	8.7 GB	11.1 GB	Runs comfortably	16K	54.1
MXFP4	4.25	8.7 GB	11.1 GB	Runs comfortably	16K	54.1
Q4_K_M	4.85	8.4 GB	10.8 GB	Runs comfortably	16K	55.8
Q4_K_S	4.58	7.9 GB	10.3 GB	Runs comfortably	16K	58.5
Q4_0	4.55	7.9 GB	10.3 GB	Runs comfortably	16K	58.8
IQ4_XS	4.25	7.4 GB	9.8 GB	Runs comfortably	16K	62.2
Q3_K_M	3.91	6.9 GB	9.3 GB	Runs comfortably	16K	66.5
IQ3_M	3.70	6.5 GB	8.9 GB	Runs comfortably	16K	69.5
IQ3_XXS	3.06	5.5 GB	7.9 GB	Runs comfortably	16K	80.5
Q2_K	2.63	4.8 GB	7.2 GB	Runs comfortably	16K	90.1
IQ2_XXS	2.06	3.9 GB	6.3 GB	Runs comfortably	16K	107
IQ1_M	1.75	3.4 GB	5.8 GB	Runs comfortably	16K	119