MiMo V2.6 Distill Qwen 9B

Parameters

9B

Context Length

262K

Modality

Text

Architecture

Dense

KV Cache Rate

128 KB/tok

License

MIT License

Release Date

21 Sept 2026

Knowledge Cutoff

-

API Pricing (per 1M)

Self-hosted only

Hardware & System Requirements

Precision

Native

Context

Workload Memory: 7.69 GB

INT4 · 8K context

Base Weights: 4.19 GB

KV Cache: 1.00 GB

Runtime & Graphs: 2.50 GB

Minimum Required Hardware

1x NVIDIA L4 24GB

7.7 / 21.6 GB usable

1x RTX 3060 12GB

7.7 / 10.8 GB usable

Apple M4 24GB

7.7 / 18.0 GB usable

Determine requirements for your specific workload:

Open in VRAM Calculator

Runtime requirements

Checkpoint format

FP16

CUDA

>= 11.8

PyTorch

>= 2.1

Transformers

>= 5.12.1

ArchitecturePrecisionAttention kernel

Blackwell, Hopper

Native FP16

FlashAttention-3

Ada Lovelace, Ampere

Native FP16

FlashAttention-2

Architecture Diagram

Evaluation Benchmarks

No evaluation benchmarks for MiMo V2.6 Distill Qwen 9B available.

Rankings

Overall Rank

-

Coding Rank

-

About MiMo V2.6 Distill Qwen 9B

MiMo V2.6 Distill Qwen 9B is a 9-billion parameter model distilled from larger MiMo reasoning models into a Qwen architecture base. It transfers complex reasoning and instruction-following performance into a lightweight deployment footprint.

Specifications

Attention

Attention Structure

Grouped-Query Attention

Attention Heads

16

Key-Value Heads

4

Attention Head Dimension

256

Position Embedding

ROPE

RoPE Theta

10,000,000

Linear Attention

Yes

Linear Attention Ratio

75.0%

Normalization

RMS Normalization

Activation Function

SwigLU

Dimensions

Base Precision

FP16

Hidden Dimension Size

4,096

Number of Layers

32

FFN Intermediate Size (Dense)

12,288

Multi-Token Prediction Heads

1

Tokenizer

Vocabulary Size

248,320

About MiMo V2

MiMo-V2-Flash is a Mixture-of-Experts (MoE) model with hybrid attention architecture designed for high-speed reasoning and agentic workflows. It features Multi-Token Prediction (MTP) to achieve state-of-the-art performance while significantly reducing inference costs. The model is optimized for long-context modeling and efficient inference.


Other MiMo V2 Models