Apple M2 Max (32GB)
The Apple M2 Max (32GB) uses unified memory architecture. For local LLM inference, macOS allocates up to 75% (24 GB) of total system RAM for GPU model weights and KV cache, with 400 GB/s unified memory bandwidth.
Memory
32 GB
Usable Memory Ceiling
24 GB (75% max allocated to GPU)
Bandwidth
400 GB/s
Max Inference Class
32B Models (Q4)
TDP
100W
VRAM Usage vs Context Length
Simulated memory requirements as context increases. Curves crossing above the reference line will cause Out Of Memory (OOM) errors.
Precision:
Model Compatibility & Speed Matrix
Simulated inference throughput and memory feasibility across popular open-weights models.
Context:
KV Cache:
Need custom layer dimensions, offloading configurations, or distributed multi-node topology?
Open in Advanced VRAM CalculatorInference Feasibility
Maximum model parameter class runnable by weight precision.
Context:
4-bit
32B Models
8-bit
14B Models
16-bit
8B Models
Fine-Tuning Feasibility
Hardware capacity for local training and adapter fine-tuning.
Context Length:
QLoRA
32B Models
LoRA
14B Models
Full Parameter Training
3B Models
Workload Recommendations
Guidance for optimal precision and operational ceilings on Apple M2 Max (32GB).
Inference Sweet Spot
Optimized for 14B models at uncompressed or Q8 precision, and 32B models at Q4 with up to 32k context.
Bandwidth & Speed Profile
With 400 GB/s aggregate bandwidth, batch size 1 inference operates in a memory-bandwidth bound regime. Generates approximately 89 tok/s on an 8B Q4 model and 11 tok/s on a 70B Q4 model.
Unified Memory Headroom
macOS limits GPU memory allocation to approximately 75% of physical unified RAM by default to ensure system stability.