Radeon AI PRO R9700 (32GB)

The Radeon AI PRO R9700 (32GB) provides 32 GB of memory with 643 GB/s bandwidth, supporting local inference via ROCm and Vulkan acceleration.

Memory

32 GB

Usable Memory Ceiling

32 GB

Bandwidth

643.2 GB/s

Max Inference Class

32B Models (Q4)

TDP

300W

VRAM Scaling by Context Length

Estimated memory requirements as context length increases. Curves crossing above the reference line indicate Out of Memory (OOM).

DeepSeek-R1 7B (7B)Sahabat-AI-Gemma2-9B (9.2B)Qwen 3.8 27B (27B)Qwen3 Next 80B A3B (80B (79B active))

Precision:

Best Models to Run on Radeon AI PRO R9700 (32GB)

Inference throughput, memory compatibility, and generation speeds for open-weights LLMs on Radeon AI PRO R9700 (32GB).

Sort by:

Context:

KV Cache:

GPUs:

RankModelParametersPrecisionTPS (Approx.)TTFTCan Run

#51

Inkling 2 Small

2B (17B active)

INT4
Q8
FP16

48 tok/s

29 tok/s

15 tok/s

~1.1s

~1.1s

~1.1s

#55

Qwen 3.8 27B

27B

INT4
Q8

32 tok/s

18 tok/s

~1.8s

~2.0s

#99

Qwen3.5-27B

27B

INT4
Q8

32 tok/s

18 tok/s

~1.8s

~2.0s

#106

Sarvam-30B

32B (2.4B active)

INT4

136 tok/s

~252 ms

#112

Qwen3.6 35B A3B

35B (3B active)

INT4

125 tok/s

~293 ms

#115

Phi-4 Reasoning Plus

14B

INT4
Q8
FP16

41 tok/s

28 tok/s

15 tok/s

~954 ms

~965 ms

~1.2s

#116

MiMo V2 Flash

15B (309B active)

INT4
Q8

3 tok/s

2 tok/s

~19.6s

~19.8s

#117

Muse Glimmer 30B

30B

INT4
Q8

29 tok/s

15 tok/s

~2.0s

~2.4s

#122

Gemma 4 26B A4B

25.2B (3.8B active)

INT4
Q8

117 tok/s

82 tok/s

~328 ms

~359 ms

#127

K2 Horizon MoVA 36B A4B

36B (4B active)

INT4

111 tok/s

~358 ms

#129

Qwen3.5-9B

9B

INT4
Q8
FP16

76 tok/s

49 tok/s

27 tok/s

~617 ms

~625 ms

~641 ms

#130

Qwen3.5-35B-A3B

35B (3B active)

INT4

125 tok/s

~293 ms

#141

Gemma 4 12B

11.95B

INT4
Q8
FP16

46 tok/s

32 tok/s

18 tok/s

~818 ms

~828 ms

~921 ms

#145

Qwen3.5-4B

4B

INT4
Q8
FP16

127 tok/s

89 tok/s

54 tok/s

~288 ms

~291 ms

~298 ms

#150

GPT-OSS 20B

21B (3.6B active)

INT4
Q8

62 tok/s

52 tok/s

~315 ms

~343 ms

#153

K2 Horizon 7B

7B

INT4
Q8
FP16

91 tok/s

60 tok/s

34 tok/s

~485 ms

~491 ms

~503 ms

#157

NVIDIA Nemotron 3 Nano 30B-A3B

3.5B (30B active)

INT4
Q8
FP16

27 tok/s

16 tok/s

9 tok/s

~1.9s

~2.0s

~2.0s

#159

Qwen3-30B-A3B

30B (3B active)

INT4
Q8

127 tok/s

88 tok/s

~287 ms

~338 ms

#164

Granite 4.2 8B

8B

INT4
Q8
FP16

83 tok/s

54 tok/s

30 tok/s

~550 ms

~557 ms

~572 ms

#167

Gemma 4 E4B

8B

INT4
Q8
FP16

83 tok/s

54 tok/s

30 tok/s

~550 ms

~557 ms

~572 ms

Inference Capacity

Maximum model parameter class runnable by weight precision.

Context:

4-bit

Q4

32B Models

8-bit

Q8

14B Models

16-bit

FP16

14B Models

Fine-Tuning Capacity

Hardware capacity for local training and adapter fine-tuning.

Context Length:

QLoRA

4-bit

32B Models

LoRA

16-bit

14B Models

Full Parameter Training

3B Models

Workload Recommendations

Guidance for optimal precision and operational ceilings on Radeon AI PRO R9700 (32GB).

Inference Sweet Spot

Optimized for 14B models at uncompressed or Q8 precision, and 32B models at Q4 with up to 32k context.

Bandwidth & Speed Profile

With 643.2 GB/s aggregate bandwidth, batch size 1 inference operates in a memory-bandwidth bound regime. Generates approximately 143 tok/s on an 8B Q4 model and 17 tok/s on a 70B Q4 model.

Frequently Asked Questions