Kimi K2-Base
Active Parameters
1T
Context Length
128K
Modality
Text
Architecture
Mixture of Experts
KV Cache Rate
69 KB/tok
License
Modified MIT License
Release Date
11 Jul 2025
Knowledge Cutoff
-
API Pricing (per 1M)
Input: $0.60 · Output: $2.50
Hardware Sizing
Precision
Context
Workload Memory: 1938.09 GB
FP16 · 8K context
Base Weights: 1922.25 GB
KV Cache: 13.34 GB
Runtime & Graphs: 2.50 GB
Minimum Required Hardware
Cluster: 16x NVIDIA H200 141GB
1938.1 / 2030.4 GB usable · TP 8 · PP 2
Determine requirements for datacenter hardware:
Upgrade to Pro
Architecture Diagram
Evaluation Benchmarks
Rank
#131
| Benchmark | Score | Rank |
|---|---|---|
Software Engineering | 0.71 | 13 |
Professional Knowledge | 0.692 | 44 |
General Text | 1419 | 97 |
Graduate-Level QA | 0.481 | 108 |
Intelligence Index | 0.13 | 215 |
General Knowledge Reference | 0.878 | 6 |
Rankings
Overall Rank
#131
Coding Rank
#54
About Kimi K2-Base
Kimi K2-Base is Moonshot AI's foundational 1T-parameter Mixture-of-Experts base model activating 32B parameters per token. Pretrained on 15.5T tokens, it provides a highly capable platform for fine-tuning complex reasoning and agentic workflows.
Technical Specifications
Attention
Attention Structure
Multi-Layer Attention
Attention Heads
64
Key-Value Heads
64
Position Embedding
ROPE
RoPE Theta
50,000
Normalization
RMS Normalization
Activation Function
SwigLU
Dimensions
Hidden Dimension Size
7,168
Number of Layers
61
FFN Intermediate Size (Dense)
2,048
Multi-Token Prediction Heads
0
Tokenizer
Vocabulary Size
163,840
Mixture of Experts
Total Expert Parameters
32.0B
Number of Experts
384
Active Experts
8
Shared Experts
1
FFN Intermediate Size (per Expert)
2,048
Dense Layers Before MoE
1
About Kimi K2
Moonshot AI's Kimi K2 is a Mixture-of-Experts model featuring one trillion total parameters, activating 32 billion per token. Designed for agentic intelligence, it utilizes a sparse architecture with 384 experts and the MuonClip optimizer for training stability, supporting a 128K token context window.