K2 Horizon 375B A23B

Open Source

Open Weights

Active Parameters

375B

Context Length

524K

Modality

Text

Architecture

Mixture of Experts (MoE)

License

Apache 2.0

Release Date

1 Sept 2026

Knowledge Cutoff

API Pricing (per 1M)

Self-hosted only

System Requirements

VRAM requirements for different quantization methods and context sizes

1,024 tokens

735.17 GB VRAM

Consumer

41x RTX 4090

24GB VRAM

Datacenter

11x NVIDIA A100

80GB VRAM

Apple Silicon

9x Apple M3 Max

128GB VRAM

524,288 tokens

863.02 GB VRAM

Consumer

49x RTX 4090

24GB VRAM

Datacenter

13x NVIDIA A100

80GB VRAM

Apple Silicon

10x Apple M3 Max

128GB VRAM

Architecture Diagram

Evaluation Benchmarks

Rank

#41

Benchmark	Score	Rank
Agentic Index Artificial Analysis	0.40	43
Coding Index Artificial Analysis	0.61	66
Intelligence Index Artificial Analysis	0.30	86

Rankings

Overall Rank

#41

Coding Rank

#50

About K2 Horizon 375B A23B

K2 Horizon 375B A23B is a massive Mixture-of-Experts language model activating 23B parameters per token. It is built for demanding reasoning, advanced code synthesis, and deep multi-turn conversational agents.

Technical Specifications

Attention

Attention Structure

Grouped-Query Attention

Attention Heads

Key-Value Heads

Attention Head Dimension

128

Position Embedding

ROPE

RoPE Theta

10,000,000

Sliding Window Attention

Sliding Window Size

Sliding Window Ratio

Linear Attention

Linear Attention Ratio

Normalization

RMS Normalization

Activation Function

SwigLU

Dimensions

Auxiliary Parameters

Hidden Dimension Size

6,144

Number of Layers

FFN Intermediate Size (Dense)

16,384

Multi-Token Prediction Heads

Tokenizer

Vocabulary Size

250,624

Mixture of Experts

Total Expert Parameters

23.0B

Number of Experts

192

Active Experts

Shared Experts

FFN Intermediate Size (per Expert)

1,792

Dense Layers Before MoE

Resources

Download Weights

About Kimi K2

Moonshot AI's Kimi K2 is a Mixture-of-Experts model featuring one trillion total parameters, activating 32 billion per token. Designed for agentic intelligence, it utilizes a sparse architecture with 384 experts and the MuonClip optimizer for training stability, supporting a 128K token context window.