Ministral 3 8B

The model's architecture is extensively documented in the official 'Ministral 3' technical report (January 2026). It is a dense decoder-only transformer with 34 layers, a hidden dimension of 4096, and 32 attention heads (8 KV heads using GQA). The report explicitly details the 'Cascade Distillation' methodology, where the model was derived by pruning Mistral Small 3.1 (24B) and continued training with logit distillation. It uses RoPE with YaRN for long-context (256k) and a 410M parameter frozen ViT encoder for vision.

While the training methodology (distillation) is well-explained, the actual composition of the underlying training data remains vague. The technical report mentions training on 'between 1 and 3 trillion tokens' but does not provide a specific breakdown of data sources (e.g., web, code, books) or percentages. It relies on the 'pretrained knowledge' of the parent model, whose own data composition is not fully disclosed beyond general 'publicly available datasets' claims.

The tokenizer is publicly available via the 'mistral-common' package and Hugging Face. It features a vocabulary size of 131,072 tokens (2^17), supporting over 40 languages and specialized control tokens for function calling. Technical documentation confirms the use of byte-fallback BPE, and the tokenizer configuration is fully inspectable in the model's repository.

The model's parameter counts are precisely disclosed: 8.4B for the language model and 0.4B for the vision encoder, totaling 8.8B. As a dense model, all parameters are active during inference. The architectural breakdown (layers, heads, dimensions) is clearly provided in Table 1 of the technical report, leaving no ambiguity regarding its density or structure.

Mistral AI discloses that the model family was trained on NVIDIA Hopper GPUs (H100/H200) as part of a partnership with NVIDIA. However, specific compute metrics for the 8B variant—such as total GPU hours, energy consumption, or carbon footprint—are not provided. The report emphasizes 'compute efficiency' through distillation but lacks the granular data required for a high transparency score in this pillar.

The technical report provides comprehensive results across standard benchmarks (MMLU, MATH, GPQA, etc.) and specifies few-shot settings (e.g., 5-shot for MMLU). While the internal evaluation harness is mentioned, the exact prompts and full reproduction code are not publicly hosted in a dedicated repository, though third-party results on LMArena and Artificial Analysis provide some external validation.

The model consistently identifies itself as part of the Ministral 3 family and is transparent about its versioning (v25.12). It accurately represents its multimodal and multilingual capabilities in documentation and API responses. There are no documented instances of the model claiming to be a competitor's product or misrepresenting its 8B scale.

The model is released under a clear, standard Apache 2.0 license, which is explicitly stated in the technical report, the Hugging Face model card, and official blog posts. This license allows for both commercial and non-commercial use, modification, and distribution without the restrictive 'Research License' terms seen in previous Mistral releases.

VRAM requirements are well-documented for various configurations. Official documentation notes that the 8B model fits in 12GB of VRAM in FP8 and provides guidance for deployment on consumer hardware like RTX GPUs. The impact of the 256k context window on KV cache memory is also acknowledged, with recommendations for setting max-model-len to manage footprint.

Mistral uses a date-based versioning system (e.g., 2512 for December 2025), which provides some clarity. However, while a general changelog exists for the API, detailed weight-level changelogs or formal deprecation schedules for specific checkpoints are less robust. Users have noted 'silent' updates to system prompt adherence in the past, though the 2512 release is currently the stable baseline.