AMD RX 6700 (10GB)

Configure in Calculator

The AMD RX 6700 (10GB) provides 10 GB of memory with 320 GB/s bandwidth, supporting local inference via ROCm and Vulkan acceleration.

Memory

10 GB

Usable Memory Ceiling

10 GB

Bandwidth

320 GB/s

Max Inference Class

8B Models (Q4)

TDP

175W

VRAM Usage vs Context Length

Simulated memory requirements as context increases. Curves crossing above the reference line will cause Out Of Memory (OOM) errors.

Precision:

FP16

Model Compatibility & Speed Matrix

Simulated inference throughput and memory feasibility across popular open-weights models.

Model	Parameters	Precision	TPS (Approx.)	TTFT
Llama 3.2 1B	1.2B	INT4	191 tok/s	~297 ms
Llama 3.2 1B	1.2B	Q8	143 tok/s	~298 ms
Llama 3.2 1B	1.2B	FP16	92 tok/s	~302 ms
Llama 3.2 3B	3.2B	INT4	110 tok/s	~731 ms
Llama 3.2 3B	3.2B	Q8	73 tok/s	~736 ms
Llama 3.2 3B	3.2B	FP16	41 tok/s	~746 ms
Mistral 7B	7.2B	INT4	61 tok/s	~1573 ms
Mistral 7B	7.2B	Q8	37 tok/s	~1584 ms
Mistral 7B	7.2B	FP16	-	-
Llama 3.1 8B	8.0B	INT4	56 tok/s	~1742 ms
Llama 3.1 8B	8.0B	Q8	32 tok/s	~1905 ms
Llama 3.1 8B	8.0B	FP16	-	-
Mistral Nemo 12B	12.2B	INT4	39 tok/s	~2636 ms
Mistral Nemo 12B	12.2B	Q8	-	-
Mistral Nemo 12B	12.2B	FP16	-	-
Qwen 2.5 14B	14.7B	INT4	31 tok/s	~3436 ms
Qwen 2.5 14B	14.7B	Q8	-	-
Qwen 2.5 14B	14.7B	FP16	-	-
DeepSeek R1 32B	32.5B	INT4	-	-
Llama 3.3 70B	70.6B	INT4	-	-
Qwen 2.5 72B	72.7B	INT4	-	-
DeepSeek V3 671B	671B (37B active)	INT4	-	-

Need custom layer dimensions, offloading configurations, or distributed multi-node topology?

Open in Advanced VRAM Calculator

Inference Feasibility

Maximum model parameter class runnable by weight precision.

4-bit

8B Models

8-bit

< 3B Models

16-bit

FP16

3B Models

Fine-Tuning Feasibility

Hardware capacity for local training and adapter fine-tuning.

Context Length:

QLoRA

4-bit

8B Models

LoRA

16-bit

3B Models

Full Parameter Training

< 1B Models

Workload Recommendations

Guidance for optimal precision and operational ceilings on AMD RX 6700 (10GB).

Inference Sweet Spot

Recommended for lightweight models (1B to 3B) at FP16/Q8, and 7B/8B models at aggressive 4-bit quantizations.

Bandwidth & Speed Profile

With 320 GB/s aggregate bandwidth, batch size 1 inference operates in a memory-bandwidth bound regime. Generates approximately 71 tok/s on an 8B Q4 model and 8 tok/s on a 70B Q4 model.