Instinct MI325X (256GB)

Datacenter

The Instinct MI325X (256GB) provides 256 GB of memory with 6000 GB/s bandwidth, supporting local inference via ROCm and Vulkan acceleration.

Memory

256 GB

Usable Memory Ceiling

256 GB

Bandwidth

6000 GB/s

Max Inference Class

70B+ Models (Q4)

TDP

750W

Simulated memory requirements as context increases. Curves crossing above the reference line will cause Out Of Memory (OOM) errors.

Precision:

Q4

Q8

FP16

Simulated inference throughput and memory feasibility across popular open-weights models.

Model	Parameters	Precision	TPS (Approx.)	TTFT
Llama 3.2 1B	1.2B	INT4	191 tok/s	~297 ms
Llama 3.2 1B	1.2B	Q8	143 tok/s	~298 ms
Llama 3.2 1B	1.2B	FP16	92 tok/s	~302 ms
Llama 3.2 3B	3.2B	INT4	110 tok/s	~731 ms
Llama 3.2 3B	3.2B	Q8	73 tok/s	~736 ms
Llama 3.2 3B	3.2B	FP16	41 tok/s	~746 ms
Mistral 7B	7.2B	INT4	61 tok/s	~1573 ms
Mistral 7B	7.2B	Q8	37 tok/s	~1584 ms
Mistral 7B	7.2B	FP16	-	-
Llama 3.1 8B	8.0B	INT4	56 tok/s	~1742 ms
Llama 3.1 8B	8.0B	Q8	32 tok/s	~1905 ms
Llama 3.1 8B	8.0B	FP16	-	-
Mistral Nemo 12B	12.2B	INT4	39 tok/s	~2636 ms
Mistral Nemo 12B	12.2B	Q8	-	-
Mistral Nemo 12B	12.2B	FP16	-	-
Qwen 2.5 14B	14.7B	INT4	31 tok/s	~3436 ms
Qwen 2.5 14B	14.7B	Q8	-	-
Qwen 2.5 14B	14.7B	FP16	-	-
DeepSeek R1 32B	32.5B	INT4	-	-
DeepSeek R1 32B	32.5B	Q8	-	-
DeepSeek R1 32B	32.5B	FP16	-	-
Llama 3.3 70B	70.6B	INT4	-	-
Llama 3.3 70B	70.6B	Q8	-	-
Llama 3.3 70B	70.6B	FP16	-	-
Qwen 2.5 72B	72.7B	INT4	-	-
Qwen 2.5 72B	72.7B	Q8	-	-
Qwen 2.5 72B	72.7B	FP16	-	-
DeepSeek V3 671B	671B (37B active)	INT4	-	-

Need custom layer dimensions, offloading configurations, or distributed multi-node topology?