Kimi K3
Active Parameters
2.8T
Context Length
1.05M
Modality
Multimodal
Architecture
Mixture of Experts
KV Cache Rate
105 KB/tok
License
Open Weights
Release Date
27 Jul 2026
Knowledge Cutoff
Jun 2024
API Pricing (per 1M)
Input: $3.00 · Output: $15.00
Hardware Sizing
Precision
Context
Workload Memory: 5446.50 GB
FP16 · 8K context
Base Weights: 5409.12 GB
KV Cache: 34.88 GB
Runtime & Graphs: 2.50 GB
Minimum Required Hardware
Workload exceeds 4,000 GB memory ceiling. Custom multi-node cluster orchestration required.
Open in VRAM Calculator
Determine requirements for your specific workload:
Open in VRAM Calculator
Architecture Diagram
Evaluation Benchmarks
Rank
#18
| Benchmark | Score | Rank |
|---|---|---|
Graduate-Level QA | 0.935 | 7 |
Reasoning | 0.91 | 9 |
Agentic Coding | 0.62 | 9 |
Web Development | max 1656 | 14 |
General | 0.79 | 14 |
LiveBench Average | 0.79 | 14 |
Coding Index | max 0.76 low 0.72 | 16 33 |
Coding | 0.81 | 16 |
General Text | max 1488 | 17 |
Data Analysis | 0.79 | 18 |
Agentic Index | max 0.50 low 0.29 | 21 61 |
Agent Arena | max 0.01 | 30 |
Mathematics | 0.84 | 51 |
Intelligence Index | low 0.17 | 197 |
Rankings
Overall Rank
#18
Coding Rank
#11
About Kimi K3
Kimi K3 is Moonshot AI's flagship 2.8T-parameter Mixture-of-Experts multimodal model engineered for long-horizon agentic workflows and complex reasoning. Featuring a 1M token context window powered by hybrid linear attention, it excels at autonomous coding and vision-in-the-loop tasks.
Technical Specifications
Attention
Attention Structure
Multi-Layer Attention
Attention Heads
96
Key-Value Heads
96
Attention Head Dimension
128
Position Embedding
ROPE
Linear Attention
Yes
Linear Attention Ratio
74.2%
Normalization
RMS Normalization
Activation Function
SwigLU
Dimensions
Hidden Dimension Size
7,168
Number of Layers
93
FFN Intermediate Size (Dense)
33,792
Tokenizer
Vocabulary Size
163,840
Mixture of Experts
Total Expert Parameters
104.0B
Number of Experts
896
Active Experts
16
Shared Experts
2
FFN Intermediate Size (per Expert)
3,072
Dense Layers Before MoE
1
About Kimi K3
Kimi K3 is a flagship Mixture-of-Experts (MoE) multimodal model series developed by Moonshot AI.
Other Kimi K3 Models
- No related models available