Ministral-8B-2410

The model is explicitly identified as a dense Transformer with 36 layers, 32 attention heads, and an embedding dimension of 4096. Mistral AI provides technical specifics such as the use of Grouped Query Attention (GQA) with 8 KV heads and an interleaved sliding-window attention mechanism to support its 128k context window. While the high-level architecture is well-documented in the release blog and model cards, a formal peer-reviewed technical paper detailing the specific pre-training methodology or architectural innovations beyond the summary is not publicly available.

Mistral AI provides only vague descriptions of the training data, stating it was trained on a 'large proportion of multilingual and code data' with a cutoff of June 2024. There is no public disclosure of specific data sources, no percentage breakdown of dataset components (e.g., web vs. books vs. code), and no documentation regarding data filtering, cleaning, or de-duplication processes. This falls under the 'vague marketing claims' category for data provenance.

The model uses the V3-Tekken tokenizer, which is publicly accessible via the 'mistral-common' library. The vocabulary size is clearly stated as 131,072 tokens. The tokenizer's support for 10+ languages and its specific BPE merges are verifiable through the provided configuration files on Hugging Face and GitHub. Documentation on the tokenizer's training alignment is present, though minor integration issues with third-party tools like TGI have been noted by the community.

The total parameter count is precisely disclosed as 8,019,808,256. As a dense model, all parameters are active during inference, which is clearly stated. The architectural breakdown (layers, heads, hidden dimensions) is fully provided in the model card, allowing for a complete understanding of parameter distribution across the network.

There is virtually no public information regarding the compute resources used to train Ministral-8B. Mistral AI has not disclosed GPU/TPU hours, hardware specifications used for training, training duration, or the carbon footprint of the process. The absence of this information is a significant transparency gap, typical of proprietary-leaning 'open-weight' releases.

Mistral provides a range of benchmark results (MMLU, AGIEval, HumanEval, etc.) in their release blog and model cards. However, they do not provide the exact evaluation code, specific prompts, or few-shot configurations used to achieve these scores. While some third-party verification exists on public leaderboards, the lack of official reproduction instructions or prompt transparency limits the ability to independently verify the claimed performance delta over competitors.

The model demonstrates high identity consistency, correctly identifying itself and its version (2410) in standard deployments. It does not exhibit the identity confusion seen in some fine-tuned models that claim to be GPT-4. Version tracking is clear through the '2410' suffix, and the model's capabilities/limitations are generally well-communicated in the documentation.

The model is released under the 'Mistral Research License' (MRL), which is a custom license. While the terms are clearly written and accessible, it is not a standard OSI-approved open-source license. It permits research and non-commercial use but requires a separate agreement for commercial applications. This creates a 'look but don't touch' environment for many users, and the definition of 'commercial' can be subject to interpretation, though the text is legally explicit.

Official documentation provides clear VRAM requirements for standard inference (e.g., 24GB for a single GPU in BF16). Community-driven documentation (e.g., Bartowski's GGUF quants) provides extensive details on VRAM requirements for various quantization levels (Q4, Q8, etc.) and their associated quality tradeoffs. While Mistral's own documentation is slightly less granular on quantization, the overall ecosystem provides good visibility into hardware needs.

Mistral uses a date-based versioning system (2410), which provides some clarity. However, they have a history of 'silent' updates to their API endpoints and have recently downscaled temperature parameters for this model family to 'unify behavior' without a major version bump. There is no comprehensive public changelog for the weights themselves, making it difficult to track if the underlying model has been modified since release.