MiMo V2 Flash

The model's architecture is extensively documented in a 31-page technical report and official GitHub repository. It is a sparse Mixture-of-Experts (MoE) decoder-only transformer with 309B total parameters and 15B active parameters. Key innovations like the 5:1 hybrid ratio of Sliding Window Attention (SWA) and Global Attention (GA), the 128-token window size, and the native Multi-Token Prediction (MTP) module are detailed with specific layer configurations (M=8 hybrid blocks, N=5 SWA layers per block). The pre-training methodology using FP8 mixed precision and the post-training Multi-Teacher On-Policy Distillation (MOPD) paradigm are also clearly described.

While the total token count (27 trillion) and broad categories (multilingual web, code, math, and domain-specialized corpora) are disclosed, there is no specific percentage breakdown or detailed list of sources. The filtering and cleaning methodologies are mentioned in general terms but lack the granular documentation required for a high score. No sample data or specific dataset names are provided for verification.

The tokenizer is publicly available as part of the model weights on Hugging Face, and its use is supported by major frameworks like SGLang and vLLM. However, official documentation lacks a detailed breakdown of the vocabulary size, tokenization algorithm (e.g., BPE vs. SentencePiece), or specific training data alignment. While functional, the technical specifics are less transparent than the model architecture.

Xiaomi provides exemplary transparency regarding parameter counts. They clearly distinguish between total parameters (309B) and active parameters (15B). The architectural breakdown is precise, noting the 0.33B parameters per block in the MTP module and the expert configuration (256 routed experts, 8 activated per token). This level of detail prevents the common MoE 'parameter inflation' marketing trap.

There is a significant lack of information regarding the compute resources used for training. While the use of FP8 mixed precision and modern NVIDIA GPUs is mentioned, there are no disclosures regarding total GPU/TPU hours, specific hardware cluster size, training duration, or the estimated carbon footprint. This is a major transparency gap.

The technical report provides comprehensive evaluation results across standard benchmarks (SWE-Bench Verified, AIME 2025, GPQA-Diamond, etc.) with specific settings (e.g., 3-shot for BBH). Evaluation code is partially available through the GitHub repository and integration with SGLang/vLLM recipes, though a single-click reproduction script for all reported numbers is not explicitly provided.

The model demonstrates high identity consistency, correctly identifying itself as MiMo, an AI developed by Xiaomi, in its recommended system prompts. It maintains clear versioning between the 'Base' and 'Flash' variants and is transparent about its knowledge cutoff (December 2024). There are no reported instances of the model claiming a competitor's identity.

The model weights and inference code are released under the permissive MIT license, which is clearly stated on Hugging Face and the official blog. However, the GitHub repository contains an Apache-2.0 license file, creating a minor conflict in documentation, although both are open-source. Commercial use is explicitly allowed, and the terms are generally clear.

Hardware requirements are well-documented for various use cases. Official guides specify VRAM needs for FP16 (618GB) and FP8, as well as consumer-grade requirements for quantized versions (e.g., 32GB VRAM for IQ3_XS). The impact of context length on memory (KV-cache reduction via SWA) is also technically explained, providing users with realistic deployment expectations.

The model uses a basic versioning system (V2 Flash), but there is no public, detailed changelog or semantic versioning history tracking minor updates or weight drifts. While the release is relatively new, the lack of a structured path for tracking future updates or accessing previous weight iterations limits the score.