NVIDIA B100 (192GB)
The NVIDIA B100 (192GB) is equipped with 192 GB of dedicated VRAM and 8000 GB/s memory bandwidth. It provides 100% addressable VRAM for CUDA and TensorRT-LLM runtimes, supporting full precision and quantized inference.
Memory
192 GB
Usable Memory Ceiling
192 GB
Bandwidth
8000 GB/s
Max Inference Class
70B+ Models (Q4)
TDP
1000W
VRAM Usage vs Context Length
Simulated memory requirements as context increases. Curves crossing above the reference line will cause Out Of Memory (OOM) errors.
Precision:
Model Compatibility & Speed Matrix
Simulated inference throughput and memory feasibility across popular open-weights models.
Context:
KV Cache:
GPUs:
Need custom layer dimensions, offloading configurations, or distributed multi-node topology?
Open in Advanced VRAM CalculatorInference Feasibility
Maximum model parameter class runnable by weight precision.
Context:
4-bit
70B+ Models
8-bit
70B Models
16-bit
70B Models
Fine-Tuning Feasibility
Hardware capacity for local training and adapter fine-tuning.
Context Length:
QLoRA
70B+ Models
LoRA
70B Models
Full Parameter Training
7B-8B Models
Workload Recommendations
Guidance for optimal precision and operational ceilings on NVIDIA B100 (192GB).
Inference Sweet Spot
Suitable for 70B parameter models at Q4/Q8 with extended 32k context windows, or high-throughput batching of smaller dense architectures.
Bandwidth & Speed Profile
With 8000 GB/s aggregate bandwidth, batch size 1 inference operates in a memory-bandwidth bound regime. Generates approximately 1778 tok/s on an 8B Q4 model and 211 tok/s on a 70B Q4 model.