Kimi K2 Thinking

The model's architecture is explicitly documented as a sparse Mixture-of-Experts (MoE) transformer with 61 layers and 384 experts. Technical disclosures confirm the use of Multi-Head Latent Attention (MLA) with 64 attention heads and SwiGLU activation. While it is described as a reasoning-focused variant of the Kimi K2 family, the specific pre-training methodology and architectural lineage (noted by third parties as heavily influenced by DeepSeek-V3) are partially disclosed through technical blog posts and model cards, though a formal peer-reviewed paper is absent.

Moonshot AI discloses a total training volume of 15.5 trillion tokens for the Kimi K2 family. However, specific dataset composition (e.g., exact ratios of web, code, and books) and detailed data cleaning or filtering methodologies remain largely opaque. Claims of 'high-quality' and 'diverse' data are made without providing public access to sample data or comprehensive source breakdowns, which is a significant gap in transparency.

The tokenizer is publicly accessible on Hugging Face and integrated into major inference frameworks like vLLM. It uses a Tiktoken-based BPE approach with a clearly stated vocabulary size of 163,840 tokens. Documentation includes special token IDs (BOS/EOS) and chat templates, allowing for independent verification and alignment with the model's claimed language support.

Moonshot AI provides exemplary transparency regarding parameter density. The model is clearly defined as having 1 trillion total parameters with exactly 32 billion active parameters per token (selecting 8 experts out of 384). This distinction between total and active parameters is consistently maintained across official documentation, preventing the common 'parameter inflation' marketing trap.

Limited information is available regarding the training compute. While third-party reports and news sources estimate a training cost of approximately $4.6 million and roughly 2.8 million H800 GPU hours for the base model, Moonshot AI has not officially disclosed precise hardware utilization, carbon footprint calculations, or exact training duration in their primary documentation.

The model provides detailed benchmark results (e.g., 44.9% on HLE with tools, 71.3% on SWE-Bench Verified) and some recommended API settings for reproduction (temperature 1.0, top_p 0.95). However, the full evaluation code and exact prompt sets used for these internal benchmarks are not publicly hosted in a reproducible repository, and third-party audits have raised concerns regarding the consistency of these results.

The model maintains a consistent identity as 'Kimi K2 Thinking' and correctly identifies its role as a reasoning agent. It distinguishes itself from the non-thinking 'Instruct' variants and provides version-aware responses. There are no widespread reports of the model claiming to be a competitor's product (e.g., GPT-4), though its internal awareness of its specific training cutoff is not always precise.

The model is released under a 'Modified MIT License.' While it allows for commercial use, it includes a restrictive clause requiring prominent UI attribution for entities exceeding 100 million monthly active users or $20 million in monthly revenue. This deviates from standard Open Source Definition (OSD) compliance, creating a 'semi-open' legal profile that requires careful legal review for large-scale enterprise adoption.

Hardware requirements are well-documented for various deployment scenarios. Official and community guides specify VRAM needs for the native INT4 format (~594GB for weights) and provide scaling estimates for context window usage. Guidance is provided for running the model on enterprise clusters (8x H100/H200) as well as extreme quantization paths for consumer hardware, though accuracy tradeoffs for the latter are less formally documented.

Versioning follows a basic naming convention (Kimi K2 Thinking vs. Turbo), but a detailed, public-facing semantic changelog is missing. While new iterations like Kimi K2.5 are announced, there is no formal mechanism for users to track silent updates or behavior drift in the underlying API endpoints, making it difficult to maintain long-term production stability.