The TPS formula is based on observed benchmark results from various sources. It considers how model parameters (like size and architecture) and quantization affect performance. The formula also includes scaling factors to account for the relative performance differences between various GPUs. While not a perfect method, it is one of the more practical ways to estimate performance without running actual benchmarks for every configuration.

Mixture-of-Experts (MoE) models using less VRAM during inference is a common misconception. While only a subset of "experts" are active for any given token, all experts must reside in memory (VRAM) for fast switching to maintain performance. The main resource benefit of MoE is in reducing compute requirements (also more so in training), and not memory usage. If you observe an MoE model using less VRAM, it is likely due to other factors, such as quantization, rather than the MoE architecture itself.

References:

• Mixture of Lookup Experts by Shibo Jie, Yehui Tang, Kai Han, et al. (2025)

• Towards Efficient Mixture of Experts: A Holistic Study of Compression Techniques by Shwai He, Daize Dong, Liang Ding, Ang Li (2024)

• Joint MoE Scaling Laws: Mixture of Experts Can Be Memory Efficient* by Jan Ludziejewski, Maciej Pióro, Jakub Krajewski, et al. (2025)

  (*Note: Under specific circumstances that is not to do with the MoE architecture itself.)

Ollama, by default, often uses a 4-bit quantization (Q4_K_M) for many models to make them run on consumer hardware. This calculator defaults to FP16, which is a common standard for performance benchmarks and cloud deployments, resulting in higher VRAM requirements. You can select a 4-bit quantization in the "Quantization" dropdown to see a comparable estimate.