Kimi-VL-A3B-Instruct

The model architecture is extensively documented in the official technical report and GitHub repository. It utilizes a sparse Mixture-of-Experts (MoE) language backbone called 'Moonlight' (16B total, 2.8B active), which is explicitly stated to be similar to the DeepSeek-V3 architecture. The vision component, 'MoonViT', is a native-resolution encoder initialized from SigLIP-SO-400M and modified with 2D Rotary Positional Embeddings (RoPE) to handle high-resolution inputs up to 1792x1792. The integration via a two-layer MLP projector and the use of FlashAttention-compatible variable-length sequence attention are well-detailed.

Moonshot AI provides a high-level breakdown of the training stages and token counts (5.2T text tokens for Moonlight pre-training, followed by 2.3T tokens of joint multimodal and text data). They disclose specific ratios for certain stages, such as upsampling long-context data to 25% during the activation phase. However, the exact sources of the 'pure-text' and 'multimodal' datasets remain described in general terms (e.g., 'curated set of multimodal instruction-response pairs') without a detailed list of public or proprietary sources, which limits full transparency.

The tokenizer is publicly accessible via the Hugging Face repository ('tiktoken.model' and 'tokenization_moonshot.py'). It is a Byte Pair Encoding (BPE) tokenizer with a known vocabulary size and clear implementation details within the provided code. The repository includes a 'chat_template.jinja' file, ensuring transparency in how special tokens and conversation formats (ChatML) are handled.

The model's parameter density is clearly and consistently disclosed across all official documentation. It is specified as a sparse MoE model with 16 billion total parameters and approximately 2.8 billion active parameters per token. The technical report further details the routing mechanism (8 experts selected from 384) and the architectural breakdown between the vision encoder and the language decoder.

Information regarding the specific training compute is conspicuously absent. While the technical report mentions the use of the 'Muon' optimizer for efficiency, it does not disclose the total GPU/TPU hours, the specific hardware cluster used for the Kimi-VL-A3B variant, or the carbon footprint. The lack of these metrics makes it impossible to verify the environmental impact or the exact resource intensity of the training process.

The technical report provides comprehensive results across a wide array of standard benchmarks (MMMU, MathVista, OSWorld, etc.) and specifies the evaluation settings (e.g., Temperature=0.0 for Instruct models). While the evaluation code is partially available in the GitHub repository, the exact prompts and few-shot examples for every benchmark are not fully itemized in a way that allows for push-button reproduction by third parties.

The model demonstrates high identity consistency, correctly identifying itself as a Moonshot AI product and maintaining version awareness (distinguishing between the 'Instruct' and 'Thinking' variants). Documentation clearly outlines the capabilities and intended use cases for each variant, and there are no documented instances of the model claiming to be a competitor's product.

The model is released under the highly permissive MIT License, which is clearly stated on the Hugging Face model card and in the GitHub repository. This license explicitly allows for commercial use, modification, and distribution. There are no conflicting terms or restrictive 'open weights' clauses that deviate from standard open-source definitions.

Official documentation and community resources provide clear guidance on VRAM requirements for deployment. For example, it is noted that approximately 42GB of VRAM is required for FP16 inference at a 1K context window, recommending dual RTX 4090s or an A6000. The impact of quantization (INT8/INT4) is also discussed in community-led documentation, though official Moonshot documentation could be more explicit about accuracy-performance tradeoffs for these quantizations.

Moonshot AI uses a form of versioning (e.g., the '2506' suffix for updated variants), and the Hugging Face repository maintains a commit history. However, there is no formal, centralized changelog that details specific behavioral changes or performance drift between minor updates. The transition from the original Kimi-VL to the 'Thinking' and '2506' versions is documented, but more granular tracking of weight updates is missing.