Kimi K3.1 Pro
Active Parameters
2.8T
Context Length
1.05M
Modality
Text
Architecture
Mixture of Experts
KV Cache Rate
4464 KB/tok
License
other
Release Date
2 Nov 2026
Knowledge Cutoff
-
API Pricing (per 1M)
Self-hosted only
Hardware Sizing
Precision
Context
Workload Memory: 5446.50 GB
FP16 · 8K context
Base Weights: 5409.12 GB
KV Cache: 34.88 GB
Runtime & Graphs: 2.50 GB
Minimum Required Hardware
Workload exceeds 4,000 GB memory ceiling. Custom multi-node cluster orchestration required.
Open in VRAM Calculator
Determine requirements for your specific workload:
Open in VRAM Calculator
Architecture Diagram
Evaluation Benchmarks
No evaluation benchmarks for Kimi K3.1 Pro available.
Rankings
Overall Rank
-
Coding Rank
-
About Kimi K3.1 Pro
Kimi K3.1 Pro is an open-weight, natively multimodal Mixture-of-Experts model featuring 2.8T total parameters with 104B active per token. Equipped with a 1-million-token context window, it excels in agentic workflows, coding, 3D creation, and financial research.
Technical Specifications
Attention
Attention Structure
DeepSeek Sparse Attention
Attention Heads
96
Key-Value Heads
96
Attention Head Dimension
128
Position Embedding
ROPE
Linear Attention
Yes
Linear Attention Ratio
74.2%
KV Latent Dimension
576
Indexer Key Dimension
128
Normalization
RMS Normalization
Activation Function
SwigLU
Dimensions
Hidden Dimension Size
7,168
Number of Layers
93
FFN Intermediate Size (Dense)
33,792
Multi-Token Prediction Heads
0
Tokenizer
Vocabulary Size
163,840
Mixture of Experts
Total Expert Parameters
104.0B
Number of Experts
896
Active Experts
16
Shared Experts
2
FFN Intermediate Size (per Expert)
3,072
Dense Layers Before MoE
1
Resources
About Kimi K3
Kimi K3 is a flagship Mixture-of-Experts (MoE) multimodal model series developed by Moonshot AI.