The TPS formula is based on observed benchmark results from various sources. It considers how model parameters (like size and architecture) and quantization affect performance. The formula also includes scaling factors to account for the relative performance differences between various GPUs. While not a perfect method, it is one of the more practical ways to estimate performance without running actual benchmarks for every configuration.

It's a common misconception that Mixture-of-Experts (MoE) models use less VRAM during inference. While only a subset of "experts" are active for any given token, all experts must reside in memory (VRAM) to enable fast switching and maintain performance. The main resource benefit of MoE is in reducing compute requirements (especially during training), not memory usage. However, MoE models do provide better generation speed compared to dense models with the same memory footprint, since they activate fewer parameters per token. If you observe an MoE model using less VRAM, it is likely due to other factors, such as quantization, rather than the MoE architecture itself.

References:

• Mixture of Lookup Experts by Shibo Jie, Yehui Tang, Kai Han, et al. (2025)

• Towards Efficient Mixture of Experts: A Holistic Study of Compression Techniques by Shwai He, Daize Dong, Liang Ding, Ang Li (2024)

• Joint MoE Scaling Laws: Mixture of Experts Can Be Memory Efficient* by Jan Ludziejewski, Maciej Pióro, Jakub Krajewski, et al. (2025)

  (*Note: The memory efficiency discussed in this paper is achieved through specific techniques and optimizations, not from the MoE architecture itself.)

Ollama, by default, often uses a 4-bit quantization (Q4_K_M) for many models to make them run on consumer hardware. This calculator defaults to FP16, which is a common standard for performance benchmarks and cloud deployments, resulting in higher VRAM requirements. You can select a 4-bit quantization in the "Quantization" dropdown to see a comparable estimate.