The AMD RX 9060 XT (16GB) provides 16 GB of memory with 320 GB/s bandwidth, supporting local inference via ROCm and Vulkan acceleration.
Memory
16 GB
Usable Memory Ceiling
16 GB
Bandwidth
320 GB/s
Max Inference Class
14B Models (Q4)
TDP
180W
Estimated memory requirements as context length increases. Curves crossing above the reference line indicate Out of Memory (OOM).
Precision:
Inference throughput, memory compatibility, and generation speeds for open-weights LLMs on AMD RX 9060 XT (16GB).
Sort by:
Context:
KV Cache:
GPUs:
| Rank | Model | Parameters | Precision | TPS (Approx.) | TTFT | Can Run |
|---|---|---|---|---|---|---|
#51 | Inkling 2 Small | 2B (17B active) | INT4 Q8 FP16 | 26 tok/s 15 tok/s 8 tok/s | ~1.7s ~1.8s ~1.8s | |
#55 | Qwen 3.8 27B | 27B | INT4 | 15 tok/s | ~3.3s | |
#99 | Qwen3.5-27B | 27B | INT4 | 15 tok/s | ~3.3s | |
#115 | Phi-4 Reasoning Plus | 14B | INT4 | 22 tok/s | ~1.5s | |
#116 | MiMo V2 Flash | 15B (309B active) | INT4 | 1 tok/s | ~33.7s | |
#122 | Gemma 4 26B A4B | 25.2B (3.8B active) | INT4 | 70 tok/s | ~558 ms | |
#129 | Qwen3.5-9B | 9B | INT4 Q8 | 44 tok/s 26 tok/s | ~973 ms ~988 ms | |
#141 | Gemma 4 12B | 11.95B | INT4 Q8 | 25 tok/s 15 tok/s | ~1.3s ~1.5s | |
#145 | Qwen3.5-4B | 4B | INT4 Q8 FP16 | 81 tok/s 53 tok/s 30 tok/s | ~450 ms ~457 ms ~472 ms | |
#150 | GPT-OSS 20B | 21B (3.6B active) | INT4 | 31 tok/s | ~582 ms | |
#153 | K2 Horizon 7B | 7B | INT4 Q8 FP16 | 54 tok/s 33 tok/s 16 tok/s | ~763 ms ~775 ms ~936 ms | |
#157 | NVIDIA Nemotron 3 Nano 30B-A3B | 3.5B (30B active) | INT4 Q8 FP16 | 14 tok/s 8 tok/s 4 tok/s | ~3.1s ~3.1s ~3.2s | |
#164 | Granite 4.2 8B | 8B | INT4 Q8 | 48 tok/s 29 tok/s | ~867 ms ~880 ms | |
#167 | Gemma 4 E4B | 8B | INT4 Q8 | 48 tok/s 29 tok/s | ~867 ms ~880 ms | |
#172 | Phi-4 | 14B | INT4 Q8 | 30 tok/s 16 tok/s | ~1.5s ~1.8s | |
#173 | Gemma 3 27B | 27B | INT4 | 15 tok/s | ~3.3s | |
#185 | DeepSeek-R1 14B | 14B | INT4 Q8 | 31 tok/s 16 tok/s | ~1.5s ~1.8s | |
#189 | Ministral 3 14B | 14B | INT4 | 22 tok/s | ~1.5s | |
#192 | Gemma 3 12B | 12B | INT4 Q8 | 34 tok/s 19 tok/s | ~1.3s ~1.4s | |
#194 | Mistral-Small-2501 | 24B | INT4 | 18 tok/s | ~2.8s | |
#196 | Ministral 3 8B | 8B | INT4 Q8 | 35 tok/s 24 tok/s | ~875 ms ~888 ms | |
#197 | OLMo 3 7B Think | 7B | INT4 Q8 | 38 tok/s 27 tok/s | ~771 ms ~782 ms | |
#198 | Qwen3.5-2B | 2B | INT4 Q8 FP16 | 122 tok/s 87 tok/s 53 tok/s | ~243 ms ~246 ms ~254 ms | |
#199 | MiniCPM5-2B | 2B | INT4 Q8 FP16 | 122 tok/s 87 tok/s 53 tok/s | ~243 ms ~246 ms ~254 ms | |
#201 | Qwen2.5-14B | 14B | INT4 Q8 | 30 tok/s 16 tok/s | ~1.5s ~1.8s | |
#204 | Gemma 2 27B | 27B | INT4 | 15 tok/s | ~3.3s | |
#205 | Gemma 4 E2B | 5.1B | INT4 Q8 FP16 | 68 tok/s 43 tok/s 24 tok/s | ~565 ms ~574 ms ~592 ms | |
#209 | DeepSeek-R1 8B | 8B | INT4 Q8 | 50 tok/s 30 tok/s | ~867 ms ~880 ms | |
#211 | Qwen3-4B | 4B | INT4 Q8 FP16 | 81 tok/s 53 tok/s 30 tok/s | ~450 ms ~457 ms ~472 ms | |
#212 | Ministral 3 3B | 3B | INT4 Q8 FP16 | 65 tok/s 50 tok/s 32 tok/s | ~351 ms ~356 ms ~367 ms | |
#214 | Qwen3.5-0.8B | 0.8B | INT4 Q8 FP16 | 186 tok/s 149 tok/s 104 tok/s | ~104 ms ~105 ms ~108 ms | |
#215 | LFM3 1.7B | 1.7B | INT4 Q8 FP16 | 134 tok/s 97 tok/s 60 tok/s | ~209 ms ~212 ms ~218 ms | |
#216 | Qwen3-14B | 14B | INT4 Q8 | 30 tok/s 16 tok/s | ~1.5s ~1.8s | |
#217 | Qwen3-8B | 8B | INT4 Q8 | 48 tok/s 29 tok/s | ~867 ms ~880 ms | |
#218 | Gemma 3 4B | 4B | INT4 Q8 FP16 | 81 tok/s 53 tok/s 30 tok/s | ~450 ms ~457 ms ~472 ms | |
#219 | Qwen3-1.7B | 1.7B | INT4 Q8 FP16 | 134 tok/s 97 tok/s 60 tok/s | ~209 ms ~212 ms ~218 ms | |
#220 | Gemma 2 9B | 9B | INT4 Q8 | 44 tok/s 26 tok/s | ~973 ms ~988 ms | |
#221 | Gemma 3 270M | 0.27B | INT4 Q8 FP16 | 218 tok/s 200 tok/s 167 tok/s | ~40 ms ~41 ms ~42 ms | |
#222 | DeepSeek-R1 7B | 7B | INT4 Q8 FP16 | 55 tok/s 34 tok/s 17 tok/s | ~763 ms ~774 ms ~867 ms | |
#223 | Qwen3-0.6B | 0.6B | INT4 Q8 FP16 | 205 tok/s 170 tok/s 124 tok/s | ~79 ms ~80 ms ~82 ms | |
#225 | Llama 3.1 8B | 8B | INT4 Q8 | 48 tok/s 29 tok/s | ~867 ms ~880 ms | |
#228 | Llama 3 8B | 8B | INT4 Q8 | 48 tok/s 29 tok/s | ~867 ms ~880 ms | |
#229 | Phi-4-Mini | 3.8B | INT4 Q8 FP16 | 84 tok/s 55 tok/s 31 tok/s | ~430 ms ~436 ms ~450 ms | |
#231 | Qwen2.5-7B | 7B | INT4 Q8 FP16 | 54 tok/s 33 tok/s 16 tok/s | ~763 ms ~775 ms ~936 ms | |
#232 | DeepSeek-R1 1.5B | 1.5B | INT4 Q8 FP16 | 142 tok/s 105 tok/s 67 tok/s | ~186 ms ~188 ms ~194 ms | |
#233 | LFM3 VL 2B | 2B | INT4 Q8 FP16 | 122 tok/s 87 tok/s 53 tok/s | ~243 ms ~246 ms ~254 ms | |
#234 | Ministral-8B-2410 | 8B | INT4 Q8 | 48 tok/s 29 tok/s | ~867 ms ~880 ms | |
#235 | Llama 3.2 3B | 3B | INT4 Q8 FP16 | 98 tok/s 66 tok/s 38 tok/s | ~346 ms ~351 ms ~362 ms | |
#237 | Phi-3-mini | 3.8B | INT4 Q8 FP16 | 84 tok/s 55 tok/s 31 tok/s | ~430 ms ~436 ms ~450 ms | |
#238 | Gemma 3n E2B IT | 6B (2B active) | INT4 Q8 FP16 | 65 tok/s 55 tok/s 37 tok/s | ~266 ms ~268 ms ~298 ms | |
#239 | Mistral-7B-Instruct-v0.1 | 7.3B | INT4 Q8 FP16 | 52 tok/s 32 tok/s 15 tok/s | ~794 ms ~806 ms ~974 ms | |
#240 | Llama 3.2 1B | 1B | INT4 Q8 FP16 | 169 tok/s 133 tok/s 89 tok/s | ~130 ms ~131 ms ~135 ms | |
#241 | Gemma 2 2B | 2B | INT4 Q8 FP16 | 122 tok/s 87 tok/s 53 tok/s | ~243 ms ~246 ms ~254 ms | |
#242 | Phi-3-medium | 14B | INT4 Q8 | 30 tok/s 16 tok/s | ~1.5s ~1.8s | |
#243 | OLMo 3 7B Instruct | 7B | INT4 Q8 | 38 tok/s 27 tok/s | ~771 ms ~782 ms | |
#245 | Qwen2-7B | 7B | INT4 Q8 FP16 | 54 tok/s 33 tok/s 16 tok/s | ~763 ms ~775 ms ~936 ms | |
#246 | Phi-3-small | 7B | INT4 Q8 FP16 | 54 tok/s 33 tok/s 16 tok/s | ~763 ms ~775 ms ~936 ms | |
#247 | Gemma 3 1B | 1B | INT4 Q8 FP16 | 169 tok/s 133 tok/s 89 tok/s | ~130 ms ~131 ms ~135 ms | |
#248 | Mistral-7B-Instruct-v0.2 | 7.3B | INT4 Q8 FP16 | 52 tok/s 32 tok/s 15 tok/s | ~794 ms ~806 ms ~974 ms | |
#250 | Gemma 1 7B | 7B | INT4 Q8 | 38 tok/s 27 tok/s | ~771 ms ~782 ms | |
#251 | Gemma 1 2B | 2B | INT4 Q8 FP16 | 131 tok/s 91 tok/s 54 tok/s | ~242 ms ~246 ms ~253 ms | |
#252 | ChatGLM3-6B | 6B | INT4 Q8 FP16 | 43 tok/s 30 tok/s 17 tok/s | ~666 ms ~675 ms ~755 ms | |
#253 | ChatGLM2-6B | 6B | INT4 Q8 FP16 | 43 tok/s 30 tok/s 17 tok/s | ~666 ms ~675 ms ~755 ms | |
#254 | ChatGLM-6B | 6B | INT4 Q8 FP16 | 43 tok/s 30 tok/s 17 tok/s | ~666 ms ~675 ms ~755 ms | |
Unranked Models (Sorted by Speed) | ||||||
- | SmolVLM2 256M Video Instruct | 0.26B | INT4 Q8 FP16 | 245 tok/s 221 tok/s 183 tok/s | ~39 ms ~39 ms ~40 ms | |
- | DINOv3 ViT-L/16 | 0.3B | INT4 Q8 FP16 | 239 tok/s 214 tok/s 173 tok/s | ~43 ms ~44 ms ~45 ms | |
- | Qwen2-0.5B | 0.5B | INT4 Q8 FP16 | 215 tok/s 183 tok/s 137 tok/s | ~67 ms ~68 ms ~69 ms | |
- | Qwen2.5-0.5B | 0.5B | INT4 Q8 FP16 | 215 tok/s 183 tok/s 137 tok/s | ~67 ms ~68 ms ~69 ms | |
- | ERNIE-4.5-0.3B | 0.3B | INT4 Q8 FP16 | 211 tok/s 193 tok/s 159 tok/s | ~44 ms ~44 ms ~46 ms | |
- | ERNIE-4.5-0.3B-Base | 0.3B | INT4 Q8 FP16 | 211 tok/s 193 tok/s 159 tok/s | ~44 ms ~44 ms ~46 ms | |
- | Liquid D1-Omni 600M | 0.6B | INT4 Q8 FP16 | 205 tok/s 170 tok/s 124 tok/s | ~79 ms ~80 ms ~82 ms | |
- | PixArt-α XL 2 1024-MS | 0.61B | INT4 Q8 FP16 | 204 tok/s 169 tok/s 123 tok/s | ~80 ms ~81 ms ~83 ms | |
- | PixArt-Σ XL 2 1024-MS | 0.61B | INT4 Q8 FP16 | 204 tok/s 169 tok/s 123 tok/s | ~80 ms ~81 ms ~83 ms | |
- | Nemotron 3.5 Parse | 0.63B | INT4 Q8 FP16 | 201 tok/s 167 tok/s 120 tok/s | ~83 ms ~84 ms ~86 ms | |
- | SD 2.1 Turbo | 0.86B | INT4 Q8 FP16 | 181 tok/s 144 tok/s 99 tok/s | ~111 ms ~113 ms ~116 ms | |
- | Stable Diffusion 1.4 | 0.86B | INT4 Q8 FP16 | 181 tok/s 144 tok/s 99 tok/s | ~111 ms ~113 ms ~116 ms | |
- | Stable Diffusion 2.1 | 0.86B | INT4 Q8 FP16 | 181 tok/s 144 tok/s 99 tok/s | ~111 ms ~113 ms ~116 ms | |
- | Falcon-1B | 1B | INT4 Q8 FP16 | 179 tok/s 138 tok/s 91 tok/s | ~129 ms ~131 ms ~135 ms | |
- | Falcon3-1B | 1B | INT4 Q8 FP16 | 169 tok/s 133 tok/s 89 tok/s | ~130 ms ~131 ms ~135 ms | |
- | MiniCPM-V 4.7 1B | 1B | INT4 Q8 FP16 | 169 tok/s 133 tok/s 89 tok/s | ~130 ms ~131 ms ~135 ms | |
- | Kandinsky 2.2 Decoder | 1.22B | INT4 Q8 FP16 | 156 tok/s 119 tok/s 78 tok/s | ~154 ms ~156 ms ~161 ms | |
- | Wan 2.1 T2V 1.3B | 1.3B | INT4 Q8 FP16 | 152 tok/s 115 tok/s 74 tok/s | ~163 ms ~165 ms ~170 ms | |
- | Qwen2-1.5B | 1.5B | INT4 Q8 FP16 | 142 tok/s 105 tok/s 67 tok/s | ~186 ms ~188 ms ~194 ms | |
- | Qwen2.5-1.5B | 1.5B | INT4 Q8 FP16 | 142 tok/s 105 tok/s 67 tok/s | ~186 ms ~188 ms ~194 ms | |
- | SVD Image-to-Video XT | 1.5B | INT4 Q8 FP16 | 142 tok/s 105 tok/s 67 tok/s | ~186 ms ~188 ms ~194 ms | |
- | SVD Image-to-Video XT 1.1 | 1.5B | INT4 Q8 FP16 | 142 tok/s 105 tok/s 67 tok/s | ~186 ms ~188 ms ~194 ms | |
- | LFM3 1.7B Base | 1.7B | INT4 Q8 FP16 | 134 tok/s 97 tok/s 60 tok/s | ~209 ms ~212 ms ~218 ms | |
- | Spark X2.5 1.7B | 1.7B | INT4 Q8 FP16 | 134 tok/s 97 tok/s 60 tok/s | ~209 ms ~212 ms ~218 ms | |
- | Hy-MT2 1.8B | 1.8B | INT4 Q8 FP16 | 130 tok/s 94 tok/s 58 tok/s | ~220 ms ~223 ms ~229 ms | |
- | Moondream 3.1 | 9B (2B active) | INT4 Q8 | 123 tok/s 87 tok/s | ~272 ms ~276 ms | |
- | CogVideoX 2B | 2B | INT4 Q8 FP16 | 122 tok/s 87 tok/s 53 tok/s | ~243 ms ~246 ms ~254 ms | |
- | LFM3 Audio 2B | 2B | INT4 Q8 FP16 | 122 tok/s 87 tok/s 53 tok/s | ~243 ms ~246 ms ~254 ms | |
- | LFM3 Omni 2B | 2B | INT4 Q8 FP16 | 122 tok/s 87 tok/s 53 tok/s | ~243 ms ~246 ms ~254 ms | |
- | Liquid D1 2B | 2B | INT4 Q8 FP16 | 122 tok/s 87 tok/s 53 tok/s | ~243 ms ~246 ms ~254 ms | |
- | LTX-Video 0.9.1 | 2B | INT4 Q8 FP16 | 122 tok/s 87 tok/s 53 tok/s | ~243 ms ~246 ms ~254 ms | |
- | Stable Diffusion 3 Medium | 2B | INT4 Q8 FP16 | 122 tok/s 87 tok/s 53 tok/s | ~243 ms ~246 ms ~254 ms | |
- | Kandinsky 3.0 | 2.1B | INT4 Q8 FP16 | 119 tok/s 84 tok/s 51 tok/s | ~253 ms ~257 ms ~265 ms | |
- | LTX-2 | 2.2B | INT4 Q8 FP16 | 117 tok/s 82 tok/s 49 tok/s | ~264 ms ~267 ms ~275 ms | |
- | SDXL 1.0 Refiner | 2.31B | INT4 Q8 FP16 | 114 tok/s 79 tok/s 47 tok/s | ~275 ms ~279 ms ~287 ms | |
- | Stable Diffusion 3.5 Medium | 2.5B | INT4 Q8 FP16 | 109 tok/s 75 tok/s 44 tok/s | ~295 ms ~299 ms ~308 ms | |
- | SDXL 1.0 Base | 2.56B | INT4 Q8 FP16 | 107 tok/s 74 tok/s 44 tok/s | ~301 ms ~305 ms ~314 ms | |
- | SDXL Turbo | 2.56B | INT4 Q8 FP16 | 107 tok/s 74 tok/s 44 tok/s | ~301 ms ~305 ms ~314 ms | |
- | Alpamayo2 Edge | 2.65B | INT4 Q8 FP16 | 105 tok/s 72 tok/s 42 tok/s | ~310 ms ~315 ms ~324 ms | |
- | Falcon-3B | 3B | INT4 Q8 FP16 | 103 tok/s 68 tok/s 39 tok/s | ~346 ms ~351 ms ~362 ms | |
- | CroissantLLM Base | 1.3B | INT4 Q8 FP16 | 102 tok/s 85 tok/s 61 tok/s | ~166 ms ~168 ms ~173 ms | |
- | Phi-1 | 1.3B | INT4 Q8 FP16 | 102 tok/s 85 tok/s 61 tok/s | ~166 ms ~168 ms ~173 ms | |
- | Phi-1.5 | 1.3B | INT4 Q8 FP16 | 102 tok/s 85 tok/s 61 tok/s | ~166 ms ~168 ms ~173 ms | |
- | DeepSeek-R1 3B | 3B | INT4 Q8 FP16 | 99 tok/s 66 tok/s 38 tok/s | ~346 ms ~351 ms ~362 ms | |
- | Mellum 2.1 12B A2.5B Thinking | 12B (2.5B active) | INT4 Q8 | 99 tok/s 67 tok/s | ~338 ms ~371 ms | |
- | Falcon3-3B | 3B | INT4 Q8 FP16 | 98 tok/s 66 tok/s 38 tok/s | ~346 ms ~351 ms ~362 ms | |
- | LFM2-VL 3B | 3B | INT4 Q8 FP16 | 98 tok/s 66 tok/s 38 tok/s | ~346 ms ~351 ms ~362 ms | |
- | Liquid D1 3B | 3B | INT4 Q8 FP16 | 98 tok/s 66 tok/s 38 tok/s | ~346 ms ~351 ms ~362 ms | |
- | Liquid D1-VL 3B | 3B | INT4 Q8 FP16 | 98 tok/s 66 tok/s 38 tok/s | ~346 ms ~351 ms ~362 ms | |
- | Liquid LFM2-VL 3B D1 | 3B | INT4 Q8 FP16 | 98 tok/s 66 tok/s 38 tok/s | ~346 ms ~351 ms ~362 ms | |
- | Qwen2.5-3B | 3B | INT4 Q8 FP16 | 98 tok/s 66 tok/s 38 tok/s | ~346 ms ~351 ms ~362 ms | |
- | LFM2.5 24B A2B | 24B (2.5B active) | INT4 | 87 tok/s | ~415 ms | |
- | ERNIE-4.5-21B-A3B | 21B (3B active) | INT4 | 81 tok/s | ~457 ms | |
- | ERNIE-4.5-21B-A3B-Base | 21B (3B active) | INT4 | 81 tok/s | ~457 ms | |
- | FLUX.2 Klein 4B | 4B | INT4 Q8 FP16 | 81 tok/s 53 tok/s 30 tok/s | ~450 ms ~457 ms ~472 ms | |
- | FrogNano 4B | 4B | INT4 Q8 FP16 | 81 tok/s 53 tok/s 30 tok/s | ~450 ms ~457 ms ~472 ms | |
- | Mage-VL Cowork | 4B | INT4 Q8 FP16 | 81 tok/s 53 tok/s 30 tok/s | ~450 ms ~457 ms ~472 ms | |
- | Spark X2.5 4B | 4B | INT4 Q8 FP16 | 81 tok/s 53 tok/s 30 tok/s | ~450 ms ~457 ms ~472 ms | |
- | EmbeddingGemma 2 | 2B | INT4 Q8 FP16 | 78 tok/s 63 tok/s 43 tok/s | ~248 ms ~251 ms ~258 ms | |
- | ERNIE-4.5-VL-28B-A3B | 28B (3B active) | INT4 | 74 tok/s | ~519 ms | |
- | ERNIE-4.5-VL-28B-A3B-Base | 28B (3B active) | INT4 | 74 tok/s | ~519 ms | |
- | CogVideoX 1.5 5B | 5B | INT4 Q8 FP16 | 69 tok/s 44 tok/s 24 tok/s | ~555 ms ~563 ms ~581 ms | |
- | CogVideoX 5B | 5B | INT4 Q8 FP16 | 69 tok/s 44 tok/s 24 tok/s | ~555 ms ~563 ms ~581 ms | |
- | Wan 2.2 TI2V 5B | 5B | INT4 Q8 FP16 | 69 tok/s 44 tok/s 24 tok/s | ~555 ms ~563 ms ~581 ms | |
- | Phi-2 | 2.7B | INT4 Q8 FP16 | 68 tok/s 53 tok/s 35 tok/s | ~320 ms ~324 ms ~334 ms | |
- | MaLLaM-3B | 3B | INT4 Q8 FP16 | 65 tok/s 50 tok/s 32 tok/s | ~351 ms ~356 ms ~367 ms | |
- | SmolLM3 3B | 3B | INT4 Q8 FP16 | 65 tok/s 50 tok/s 32 tok/s | ~351 ms ~356 ms ~367 ms | |
- | Falcon-7B | 7B | INT4 Q8 FP16 | 56 tok/s 34 tok/s 17 tok/s | ~762 ms ~774 ms ~867 ms | |
- | AuraFlow v0.2 | 6.8B | INT4 Q8 FP16 | 55 tok/s 34 tok/s 17 tok/s | ~742 ms ~754 ms ~844 ms | |
- | AuraFlow v0.3 | 6.8B | INT4 Q8 FP16 | 55 tok/s 34 tok/s 17 tok/s | ~742 ms ~754 ms ~844 ms | |
- | Falcon3-7B | 7B | INT4 Q8 FP16 | 54 tok/s 33 tok/s 16 tok/s | ~763 ms ~775 ms ~936 ms | |
- | FastFrames-L | 7B | INT4 Q8 FP16 | 54 tok/s 33 tok/s 16 tok/s | ~763 ms ~775 ms ~936 ms | |
- | Hy-MT2 7B | 7B | INT4 Q8 FP16 | 54 tok/s 33 tok/s 16 tok/s | ~763 ms ~775 ms ~936 ms | |
- | Mistral-7B-v0.1 | 7.3B | INT4 Q8 FP16 | 52 tok/s 32 tok/s 15 tok/s | ~794 ms ~806 ms ~974 ms | |
- | Youtu Parsing Omni | 8B | INT4 Q8 | 50 tok/s 30 tok/s | ~867 ms ~880 ms | |
- | Stable Diffusion 3.5 Large | 8.1B | INT4 Q8 | 48 tok/s 29 tok/s | ~877 ms ~891 ms | |
- | Stable Diffusion 3.5 Large Turbo | 8.1B | INT4 Q8 | 48 tok/s 29 tok/s | ~877 ms ~891 ms | |
- | FLUX.2 Klein 9B | 9B | INT4 Q8 | 44 tok/s 26 tok/s | ~973 ms ~988 ms | |
- | LensVLM 9B | 9B | INT4 Q8 | 44 tok/s 26 tok/s | ~973 ms ~988 ms | |
- | MiMo V2.6 Distill Qwen 9B | 9B | INT4 Q8 | 44 tok/s 26 tok/s | ~973 ms ~988 ms | |
- | ChatGLM3-6B-32K | 6B | INT4 Q8 FP16 | 43 tok/s 30 tok/s 17 tok/s | ~666 ms ~675 ms ~755 ms | |
- | Yi-6B | 6B | INT4 Q8 FP16 | 43 tok/s 30 tok/s 17 tok/s | ~666 ms ~675 ms ~755 ms | |
- | Falcon3-10B | 10B | INT4 Q8 | 40 tok/s 24 tok/s | ~1.1s ~1.1s | |
- | Mochi 1 Preview | 10B | INT4 Q8 | 40 tok/s 24 tok/s | ~1.1s ~1.1s | |
- | Falcon2-11B | 11B | INT4 Q8 | 39 tok/s 23 tok/s | ~1.2s ~1.2s | |
- | Hunyuan Lite | 7B | INT4 Q8 | 38 tok/s 27 tok/s | ~771 ms ~782 ms | |
- | Kimi-VL-A3B-Instruct | 16B (3B active) | INT4 | 38 tok/s | ~453 ms | |
- | OLMo 3 7B Base | 7B | INT4 Q8 | 38 tok/s 27 tok/s | ~771 ms ~782 ms | |
- | SEA-LION-7B | 7.1B | INT4 Q8 | 38 tok/s 27 tok/s | ~781 ms ~792 ms | |
- | SEA-LION-7B-Instruct | 7.1B | INT4 Q8 | 38 tok/s 27 tok/s | ~781 ms ~792 ms | |
- | FLUX.1 Canny Dev | 11.9B | INT4 Q8 | 35 tok/s 20 tok/s | ~1.3s ~1.4s | |
- | FLUX.1 Depth Dev | 11.9B | INT4 Q8 | 35 tok/s 20 tok/s | ~1.3s ~1.4s | |
- | FLUX.1 Dev | 11.9B | INT4 Q8 | 35 tok/s 20 tok/s | ~1.3s ~1.4s | |
- | FLUX.1 Fill Dev | 11.9B | INT4 Q8 | 35 tok/s 20 tok/s | ~1.3s ~1.4s | |
- | FLUX.1 Schnell | 11.9B | INT4 Q8 | 35 tok/s 20 tok/s | ~1.3s ~1.4s | |
- | Sahabat-AI-Llama3-8B-Instruct | 8B | INT4 Q8 | 35 tok/s 24 tok/s | ~875 ms ~888 ms | |
- | Typhoon-2-8B | 8B | INT4 Q8 | 35 tok/s 24 tok/s | ~875 ms ~888 ms | |
- | GLM-4-9B | 9B | INT4 Q8 | 32 tok/s 21 tok/s | ~982 ms ~1.1s | |
- | GLM-4-9B-Chat | 9B | INT4 Q8 | 32 tok/s 21 tok/s | ~982 ms ~1.1s | |
- | GLM-4-9B-Chat-1M | 9B | INT4 Q8 | 32 tok/s 21 tok/s | ~982 ms ~1.1s | |
- | GLM-4V | 9B | INT4 Q8 | 32 tok/s 21 tok/s | ~982 ms ~1.1s | |
- | HunyuanVideo 1.5 720P T2V | 13B | INT4 Q8 | 32 tok/s 18 tok/s | ~1.4s ~1.5s | |
- | Yi-9B | 9B | INT4 Q8 | 32 tok/s 21 tok/s | ~982 ms ~1.1s | |
- | Sahabat-AI-Gemma2-9B | 9.2B | INT4 Q8 | 31 tok/s 20 tok/s | ~1.0s ~1.1s | |
- | Sahabat-AI-Gemma2-9B-Instruct | 9.2B | INT4 Q8 | 31 tok/s 20 tok/s | ~1.0s ~1.1s | |
- | Wan 2.1 I2V 14B 720P | 14B | INT4 Q8 | 30 tok/s 16 tok/s | ~1.5s ~1.8s | |
- | Wan 2.1 T2V 14B | 14B | INT4 Q8 | 30 tok/s 16 tok/s | ~1.5s ~1.8s | |
- | Wan 2.2 T2V A14B | 14B | INT4 Q8 | 30 tok/s 16 tok/s | ~1.5s ~1.8s | |
- | FLUX.2 Dev | 15B | INT4 Q8 | 28 tok/s 15 tok/s | ~1.6s ~1.9s | |
- | GreenMind-14B-R1 | 14B | INT4 | 22 tok/s | ~1.5s | |
- | Arete 27B | 27B | INT4 | 15 tok/s | ~3.3s | |
- | AREX-2 | 27B | INT4 | 15 tok/s | ~3.3s | |
- | PPLX Decider v1 27B | 27B | INT4 | 15 tok/s | ~3.3s | |
- | PPLX Decider v1.1 27B | 27B | INT4 | 15 tok/s | ~3.3s | |
- | PPLX Decider v1.3 | 27B | INT4 | 15 tok/s | ~3.3s | |
Maximum model parameter class runnable by weight precision.
Context:
4-bit
14B Models
8-bit
8B Models
16-bit
3B Models
Hardware capacity for local training and adapter fine-tuning.
Context Length:
QLoRA
14B Models
LoRA
8B Models
Full Parameter Training
1B Models
Guidance for optimal precision and operational ceilings on AMD RX 9060 XT (16GB).
Inference Sweet Spot
Optimal for 8B models across extended context (32k+). 14B models fit comfortably at Q4 precision with standard context.
Bandwidth & Speed Profile
With 320 GB/s aggregate bandwidth, batch size 1 inference operates in a memory-bandwidth bound regime. Generates approximately 71 tok/s on an 8B Q4 model and 8 tok/s on a 70B Q4 model.