Radeon PRO W7600 (8GB)

The Radeon PRO W7600 (8GB) provides 8 GB of memory with 288 GB/s bandwidth, supporting local inference via ROCm and Vulkan acceleration.

Memory

8 GB

Usable Memory Ceiling

8 GB

Bandwidth

288 GB/s

Max Inference Class

8B Models (Q4)

TDP

130W

VRAM Scaling by Context Length

Estimated memory requirements as context length increases. Curves crossing above the reference line indicate Out of Memory (OOM).

SmolVLM2 256M Video Instruct (0.26B)SDXL Turbo (2.56B)Mistral-7B-Instruct-v0.2 (7.3B)ERNIE-4.5-21B-A3B-Base (21B (3B active))

Precision:

Best Models to Run on Radeon PRO W7600 (8GB)

Inference throughput, memory compatibility, and generation speeds for open-weights LLMs on Radeon PRO W7600 (8GB).

Sort by:

Context:

KV Cache:

GPUs:

RankModelParametersPrecisionTPS (Approx.)TTFTCan Run

#51

Inkling 2 Small

2B (17B active)

INT4
Q8
FP16

24 tok/s

14 tok/s

7 tok/s

~5.1s

~5.2s

~5.2s

#129

Qwen3.5-9B

9B

INT4

40 tok/s

~2.9s

#145

Qwen3.5-4B

4B

INT4
Q8

75 tok/s

48 tok/s

~1.3s

~1.3s

#153

K2 Horizon 7B

7B

INT4

49 tok/s

~2.2s

#157

NVIDIA Nemotron 3 Nano 30B-A3B

3.5B (30B active)

INT4
Q8

13 tok/s

7 tok/s

~9.0s

~9.8s

#164

Granite 4.2 8B

8B

INT4

44 tok/s

~2.5s

#167

Gemma 4 E4B

8B

INT4

44 tok/s

~2.5s

#192

Gemma 3 12B

12B

INT4

28 tok/s

~4.5s

#196

Ministral 3 8B

8B

INT4

29 tok/s

~3.0s

#197

OLMo 3 7B Think

7B

INT4

33 tok/s

~2.4s

#198

Qwen3.5-2B

2B

INT4
Q8
FP16

115 tok/s

81 tok/s

49 tok/s

~699 ms

~703 ms

~711 ms

#199

MiniCPM5-2B

2B

INT4
Q8
FP16

115 tok/s

81 tok/s

49 tok/s

~699 ms

~703 ms

~711 ms

#205

Gemma 4 E2B

5.1B

INT4
Q8

63 tok/s

38 tok/s

~1.6s

~1.8s

#209

DeepSeek-R1 8B

8B

INT4

46 tok/s

~2.5s

#211

Qwen3-4B

4B

INT4
Q8

75 tok/s

48 tok/s

~1.3s

~1.3s

#212

Ministral 3 3B

3B

INT4
Q8

60 tok/s

46 tok/s

~1.0s

~1.0s

#214

Qwen3.5-0.8B

0.8B

INT4
Q8
FP16

179 tok/s

142 tok/s

97 tok/s

~288 ms

~289 ms

~292 ms

#215

LFM3 1.7B

1.7B

INT4
Q8
FP16

126 tok/s

91 tok/s

56 tok/s

~599 ms

~602 ms

~609 ms

#217

Qwen3-8B

8B

INT4

44 tok/s

~2.5s

#218

Gemma 3 4B

4B

INT4
Q8

75 tok/s

48 tok/s

~1.3s

~1.3s

Inference Capacity

Maximum model parameter class runnable by weight precision.

Context:

4-bit

Q4

8B Models

8-bit

Q8

3B Models

16-bit

FP16

< 1B Models

Fine-Tuning Capacity

Hardware capacity for local training and adapter fine-tuning.

Context Length:

QLoRA

4-bit

8B Models

LoRA

16-bit

3B Models

Full Parameter Training

< 1B Models

Workload Recommendations

Guidance for optimal precision and operational ceilings on Radeon PRO W7600 (8GB).

Inference Sweet Spot

Recommended for lightweight models (1B to 3B) at FP16/Q8, and 7B/8B models at aggressive 4-bit quantizations.

Bandwidth & Speed Profile

With 288 GB/s aggregate bandwidth, batch size 1 inference operates in a memory-bandwidth bound regime. Generates approximately 64 tok/s on an 8B Q4 model and 8 tok/s on a 70B Q4 model.

Frequently Asked Questions