Ministral 3 3B

The model's architecture is extensively documented in the 'Ministral 3' technical report (arXiv:2601.03006). It is explicitly identified as a descendant of Mistral Small 3.1, derived through a 'Cascade Distillation' process involving iterative pruning and continued training. Technical specifications are precise, detailing a 26-layer dense Transformer-based decoder (3.4B parameters) coupled with a frozen 410M parameter Vision Transformer (ViT) encoder from Mistral Small 3.1. Key architectural components such as Grouped Query Attention (GQA), Rotary Positional Embeddings (RoPE) with YaRN, and tied input-output embeddings are clearly defined and justified for the 3B scale.

While the training methodology (Cascade Distillation) is well-documented, the actual composition of the pretraining dataset remains vague. The technical report mentions training on 1 to 3 trillion tokens but does not provide a specific breakdown of data sources (e.g., percentages of web, code, or books). It refers to 'diverse open and proprietary sources' and 'high-quality multimodal data' without naming specific datasets. Filtering and cleaning procedures are mentioned as a 'rigorous multi-step pipeline' but lack granular detail beyond general categories of removed content.

The tokenizer is publicly accessible via the 'mistral-common' GitHub repository and Hugging Face. It uses a vocabulary size of 131,072 tokens, which is explicitly stated in both the technical report and the model configuration files. The tokenizer is shared across the Ministral 3 family, ensuring consistency, and supports dozens of languages as claimed. Documentation for the 'MistralCommonBackend' provides clear implementation details for developers.

The model's parameter counts are clearly and accurately disclosed. It is a dense model (not MoE), with a total of approximately 3.8B parameters, broken down into a 3.4B parameter language decoder and a 410M parameter vision encoder. The use of tied embeddings in the 3B variant is specifically noted as a design choice to manage parameter density. This transparency prevents the common confusion between total and active parameters often found in sparse architectures.

Mistral AI discloses that the models were trained on NVIDIA Hopper GPUs (specifically H200s) as part of a partnership with NVIDIA. However, the total compute budget in terms of GPU-hours is not provided. While the 'Cascade Distillation' method is highlighted as being more compute-efficient than training from scratch, specific metrics regarding energy consumption, carbon footprint, or the exact duration of the training runs are absent from the public documentation.

The technical report provides comprehensive benchmark results across standard datasets (MMLU, MATH, GPQA, etc.) and compares them against size-matched competitors like Qwen 3 and Gemma 3. While evaluation settings (e.g., 5-shot, CoT) are specified, the exact evaluation code and full prompt sets are not fully public, though some third-party verification (e.g., LMArena, independent reviewers) is available. The report acknowledges the use of an 'internal harness' for some comparisons, which limits full independent reproducibility.

The model demonstrates high identity consistency, correctly identifying itself as a Mistral AI model and distinguishing between its variants (Base, Instruct, Reasoning). Versioning is clear with the '2512' (December 2025) suffix used in official naming conventions. There are no reported instances of the model claiming to be a competitor's product or misrepresenting its 3B-scale capabilities.

The model is released under the Apache 2.0 license, which is a standard, highly permissive open-source license. This allows for commercial use, modification, and distribution without the restrictive 'non-commercial' or 'research-only' clauses found in some other 'open' models. The license terms are clearly stated on the Hugging Face model card, the official blog, and within the repository.

Hardware requirements are well-documented for various deployment scenarios. Official documentation and community guides (e.g., vLLM, Ollama) provide VRAM estimates for FP16 (approx. 8.5GB) and quantized versions (under 6GB for Q8/Q4). The impact of the 256k context window on memory scaling is also addressed, with recommendations for specific GPUs (e.g., RTX 3060, RTX 4000 Ada) provided in technical guides.

Mistral AI maintains a general changelog for its API and model releases, and the use of date-based versioning (e.g., '2512') provides some tracking. However, detailed changelogs for specific weight updates or fine-tuning iterations are less granular. While the transition from the previous 'Ministral 2410' to the '2512' version is documented, there is limited information on how minor updates or safety alignment changes might affect performance drift over time.