Kimi K2-Base

Kimi K2-Base is explicitly documented as a Mixture-of-Experts (MoE) transformer with 1.04 trillion total parameters and 32 billion active parameters. The architecture is detailed in a technical report and GitHub documentation, specifying 61 layers, 384 experts (8 selected per token), and a hidden dimension of 7168. It utilizes Multi-head Latent Attention (MLA) and the SwiGLU activation function. A significant technical disclosure is the use of the 'MuonClip' optimizer with a 'qk-clip' mechanism to stabilize training at scale, which is a high level of transparency for a foundational model.

While Moonshot AI discloses the scale of the pre-training data (15.5 trillion tokens), the specific composition and sources remain largely opaque. Documentation mentions a 'diverse mixture of web text, books, code, and multilingual content' but lacks a granular percentage breakdown or specific source names. The mention of an 'agentic data synthesis pipeline' for post-training provides some insight into the methodology for instruction tuning, but the upstream pre-training data provenance is described only in vague, high-level terms.

The tokenizer is publicly accessible via the model weights on Hugging Face and is well-documented. It features a large vocabulary size of 160,000 tokens, supporting multilingual text and code. Independent analysis by third parties (e.g., Unsloth) has verified its regex patterns and handling of Chinese characters, noting its similarity to GPT-4o's tokenizer with specific optimizations for Han characters. The EOS token and chat templates are clearly defined in the repository.

Moonshot AI provides exemplary transparency regarding parameter density. They clearly distinguish between the 1 trillion total parameters and the 32 billion active parameters per token. The documentation further breaks down the expert structure (384 total experts, 8 active per token, 1 shared expert) and provides specific dimensions for the attention and MoE layers. This prevents the common 'parameter inflation' marketing trap often seen with MoE models.

Information regarding training compute is minimal. While unofficial reports and leaks suggest a training cost of approximately $4.6 million using H800 GPUs, Moonshot AI's official stance in AMAs is that costs are 'hard to quantify.' There is no public disclosure of total GPU/TPU hours, specific hardware cluster configurations, or the carbon footprint associated with the 15.5 trillion token training run.

The model provides a comprehensive list of benchmark results (MMLU, GPQA, SWE-bench, etc.) and a technical report. However, the evaluation code is not fully public, and exact prompts for all benchmarks are not disclosed. While they provide 'Best Practices for Benchmarking' documentation with recommended settings (temperature, top_p, max_tokens), the lack of a reproducible evaluation harness and the discovery of significant contamination in related 'Thinking' variants by independent labs (e.g., ETH Zurich) necessitates a cautious score.

Kimi K2-Base maintains a consistent identity across its documentation and API. It correctly identifies its versioning (K2 vs K2.5) and its nature as a foundational model. There are no documented instances of the base model claiming to be a competitor's model or misrepresenting its core MoE architecture. The model card clearly distinguishes between the Base, Instruct, and Thinking variants.

The model is released under a 'Modified MIT License.' The license is publicly available on GitHub and is largely permissive, allowing for research and commercial use. However, it includes a specific 'attribution' clause for high-scale commercial users (>100M MAU or >$20M monthly revenue), requiring prominent display of 'Kimi K2' on the UI. While clear, this custom modification moves it away from a standard Open Source definition.

Hardware requirements are well-documented for various precisions. Official and third-party guides (e.g., ApX, RunPod) provide VRAM estimates for FP16 (~2.4TB), INT8 (~1TB), and INT4 (~600GB). The model was notably developed with 'Quantization-Aware Training' (QAT) for native INT4 support, and documentation explicitly discusses the memory trade-offs and the need for multi-GPU clusters (e.g., 8x H200/B200) to run the full 1T parameter model.

Moonshot AI uses date-based versioning (e.g., 0711, 0905) and maintains a basic changelog on Hugging Face for newer iterations like K2.5. However, for the K2-Base model, there is limited documentation regarding weight updates or silent changes. While they provide deprecation notices for certain tokens or system prompts in the Instruct/Thinking versions, the Base model's versioning history is less granular.