GPU infrastructure planning for production

Capacity modeling for teams deploying LLMs and diffusion models at scale.

Used by engineering teams from

NVIDIAGoogle CloudAmazon Web ServicesHPEDellHitachiAkamaiByteDance
Capacity modeling

Design clusters for peak throughput

Model KV cache growth, concurrency limits, and multi-node latency before you provision a single GPU.

Weights, KV cache, CUDA graphs, and runtime activation overhead
Concurrency SLAs, per-user TPS, and p50/p95 TTFT latency
Single GPU to multi-node NVLink and PCIe cluster topologies
Verified specifications

Every frontier model, updated with each new release

Evaluate the latest open-weight models and quantization footprints across datacenter GPUs, RTX workstations, and Apple Silicon.

Memory and KV scaling across dense Transformer, MoE, Mamba, and Diffusion models
Hardware profiles calibrated to inference limits
Direct parameter presets for every listed model and GPU
Physical calibration

Calibrated on physical GPU telemetry

Every memory and throughput curve is continuously calibrated against live runs across NVIDIA, AMD, and Apple Silicon hardware.

Plan clusters with measured hardware telemetry before provisioning compute.

Open-weight model inference specifications

Context windows, quantization footprints, and target GPUs across leading open-weight architectures.

ModelParametersAttentionContextMin VRAMRuns on
Kimi K3.1 Pro

Moonshot AI

2800B A104BDSA1.0M1321.7 GB32x H100
Qwen3.8 2.4T A95B

Alibaba

2400B A95BGQA262.1k1132.7 GB16x H100
GLM-5.3

Z.ai

753.9B A51.6BDSA1.0M356.4 GB8x H100
DeepSeek V4.1 Flash

DeepSeek

552B A16BCSA21.0M261.1 GB4x H100
DeepSeek-V4-Flash-Vision-Exp

DeepSeek

290.9B A20.4BDSA1.0M138.0 GB2x H100
K2 Horizon MoVA 36B A4B

Moonshot AI

36B A4BGQA524.3k17.8 GB1x RTX 4090
Qwen 3.8 27B

Alibaba

27BGQA262.1k13.2 GB1x 16GB GPU
K2 Horizon 7B

Moonshot AI

7BGQA524.3k3.9 GB1x 8GB GPU

GPU & workstation profiles

Explore hardware directory →

Engineering courses & roadmaps

150+ courses covering transformer internals, quantization kernels, distributed inference serving, and PyTorch foundations.

Hardware benchmarks & deployment guides

GPU architecture comparisons, model sizing breakdowns, and serving optimization guides.