Active Parameters
1T
Context Length
512K
Modality
Text
Architecture
Mixture of Experts (MoE)
License
Modified MIT License
Release Date
5 Feb 2026
Knowledge Cutoff
Oct 2025
Total Expert Parameters
968.0B
Number of Experts
384
Active Experts
8
Attention Structure
Multi-Head Attention
Hidden Dimension Size
7168
Number of Layers
61
Attention Heads
64
Key-Value Heads
-
Activation Function
SwigLU
Normalization
RMS Normalization
Position Embedding
Absolute Position Embedding
Kimi K2.5 is a high-capacity Mixture-of-Experts (MoE) large language model developed by Moonshot AI, designed to address complex reasoning and multimodal tasks at scale. The model is built on a massive 1-trillion parameter architecture that employs a sparse activation strategy, utilizing only 32 billion active parameters per forward pass to maintain computational efficiency while providing deep representational capacity. It distinguishes itself through its native multimodal training, where vision and language components are co-trained from the initial pre-training phase on approximately 15 trillion tokens, enabling unified processing of visual data and textual information.
Technically, Kimi K2.5 integrates several architectural innovations, most notably the use of Multi-head Latent Attention (MLA) and a specialized 384-expert MoE structure. The attention mechanism is optimized for high-throughput inference and long-context performance, supporting context windows up to 256,000 tokens. The model also introduces an 'Agent Swarm' paradigm, a self-directed multi-agent orchestration system trained via Parallel Agent Reinforcement Learning (PARL). This allows the model to decompose complex objectives into independent sub-tasks executed by up to 100 parallel sub-agents, significantly reducing serial execution latency in tool-heavy workflows.
In practical application, Kimi K2.5 functions as a versatile engine for advanced coding, document synthesis, and automated reasoning. It features four distinct operational modes, Instant, Thinking, Agent, and Agent Swarm, allowing users to balance response speed and reasoning depth based on the task requirement. Its native visual coding capabilities allow for the direct translation of UI designs and video workflows into functional code, while its extensive context window facilitates the analysis of large codebases and complex technical documentation. The model's training stability at the trillion-parameter scale is achieved through the MuonClip optimizer, which mitigates common loss spikes associated with sparse architectures.
Moonshot AI's Kimi K2 is a Mixture-of-Experts model featuring one trillion total parameters, activating 32 billion per token. Designed for agentic intelligence, it utilizes a sparse architecture with 384 experts and the MuonClip optimizer for training stability, supporting a 128K token context window.
Rank
#2
| Benchmark | Score | Rank |
|---|---|---|
Mathematics LiveBench Mathematics | 0.85 | 7 |
Reasoning LiveBench Reasoning | 0.76 | 11 |
Overall Rank
#2 🥈
Coding Rank
-
Total Score
69
/ 100
Kimi K2.5 exhibits strong transparency in its architectural specifications and hardware requirements, providing rare detail on MoE parameter density and native multimodal integration. However, it remains opaque regarding training compute resources and the specific composition of its 15-trillion token dataset. While the open-weights release and detailed technical report are commendable, the reliance on internal benchmarks for certain performance claims necessitates cautious verification.
Architectural Provenance
Kimi K2.5 is extensively documented as a 1-trillion parameter Mixture-of-Experts (MoE) model built upon the Kimi-K2-Base. The technical report (arXiv:2602.02276) and official GitHub repository provide deep architectural specifics, including the use of Multi-head Latent Attention (MLA), a 384-expert MoE structure (8 active per token), and the MuonClip optimizer for training stability. The model's 'Agent Swarm' paradigm and Parallel Agent Reinforcement Learning (PARL) methodology are also publicly detailed.
Dataset Composition
While Moonshot AI discloses the scale of the training data (15 trillion mixed visual and text tokens) and the methodology of 'joint text-vision pre-training' from the start, specific data sources and the exact percentage breakdown of the dataset remain undisclosed. The documentation mentions 'diverse internet data' and 'multimodal files' but lacks the granular transparency required for a higher score, such as specific domain proportions or detailed filtering/cleaning logs.
Tokenizer Integrity
The tokenizer is publicly accessible via the official GitHub and Hugging Face repositories. It features a vocabulary size of 160,000 tokens and uses a specialized configuration to support native multimodality (e.g., <|media_begin|> tokens). The alignment between the tokenizer and the claimed language/multimodal support is verifiable through the provided code and model cards.
Parameter Density
Moonshot AI provides exemplary transparency regarding parameter density. They explicitly state the total parameter count (1T) and the active parameters per forward pass (32B). Furthermore, the architectural breakdown is highly detailed: 61 layers (including 1 dense layer), 384 experts, 8 selected experts per token, and a 400M parameter vision encoder (MoonViT). This level of detail is rare for models of this scale.
Training Compute
Despite the detailed architectural info, there is a significant lack of transparency regarding training compute. No specific GPU/TPU hours, hardware cluster specifications, or carbon footprint calculations are provided in the technical report or official documentation. The mention of the 'MuonClip' optimizer for efficiency does not substitute for concrete compute resource disclosure.
Benchmark Reproducibility
Moonshot AI provides comprehensive evaluation results across standard (AIME 2025, MMLU-Pro) and internal (Kimi Code Bench, AI Office Bench) benchmarks. They disclose evaluation settings (temperature 1.0, top_p 0.95) and provide a technical report. However, the use of 'internally developed evaluation frameworks' for key coding tasks and the lack of full evaluation code for all reported metrics limit complete third-party reproducibility.
Identity Consistency
The model demonstrates high identity consistency, correctly identifying itself as Kimi K2.5 and distinguishing between its operational modes (Instant, Thinking, Agent, Swarm). Documentation clearly outlines versioning (K2 vs K2.5) and capabilities. There are no reported instances of the model claiming to be a competitor's product or misrepresenting its AI nature.
License Clarity
The model is released under a 'Modified MIT License.' While it allows for commercial use, modification, and distribution, it includes a specific 'attribution' clause requiring products with >100M MAU or >$20M monthly revenue to prominently display 'Kimi K2.5' on the UI. This is a clear, albeit restrictive, license that is well-documented in the GitHub repository.
Hardware Footprint
Hardware requirements are thoroughly documented for various deployment scenarios. Official guides specify VRAM needs for INT4 quantization (the native format) and provide recommended configurations for production (e.g., 8x H200/B200 nodes). Third-party documentation from Unsloth and KTransformers further details extreme compression requirements (24GB VRAM + 256GB RAM) for local execution.
Versioning Drift
Moonshot AI maintains a basic changelog on Hugging Face (e.g., fixing system prompt issues and token names). However, the model has shown signs of 'effective compression degradation' in long-context windows according to independent analysis, and there is no formal, detailed version history or public deprecation roadmap for older Kimi K2 variants.
Full Calculator
Choose the quantization method for model weights
Context Size: 1,024 tokens