Mistral Large 3
Active Parameters
41B
Context Length
256K
Modality
Multimodal
Architecture
Mixture of Experts
KV Cache Rate
352 KB/tok
License
Apache 2.0
Release Date
2 Dec 2025
Knowledge Cutoff
Oct 2024
API Pricing (per 1M)
Input: $0.50 · Output: $1.50
Hardware & System Requirements
Precision
Context
Workload Memory: 1338.90 GB
FP16 · 8K context
Base Weights: 1333.65 GB
KV Cache: 2.75 GB
Runtime & Graphs: 2.50 GB
Minimum Required Hardware
16x NVIDIA H200 141GB
1338.9 / 2030.4 GB usable · TP 8 · PP 2
Determine requirements for your specific workload:
Upgrade to Pro
Runtime requirements
Checkpoint format
FP16
CUDA
>= 11.8
PyTorch
>= 2.1
Transformers
>= 4.46.2
| Architecture | Precision | Attention kernel |
|---|---|---|
Blackwell, Hopper | Native FP16 | FlashAttention-3 |
Ada Lovelace, Ampere | Native FP16 | FlashAttention-2 |
Architecture Diagram
Evaluation Benchmarks
Rank
#164
| Benchmark | Score | Rank |
|---|---|---|
StackUnseen | 0.516 | 22 |
General Text | 1415 | 110 |
Graduate-Level QA | 0.439 | 111 |
Agentic Index | 1 | 115 |
Web Development | 1232 | 118 |
Coding Index | 0.20 | 134 |
Intelligence Index | 0.09 | 249 |
Rankings
Overall Rank
#164
Coding Rank
#126
About Mistral Large 3
Mistral Large 3 is Mistral AI's 675B flagship multimodal Mixture-of-Experts foundation model activating 41B parameters for enterprise RAG and agents. It features native vision processing, FP8/NVFP4 quantization, and a 256K token context window.
Specifications
Attention
Attention Structure
Multi-Head Attention
Attention Heads
96
Key-Value Heads
8
Attention Head Dimension
128
Position Embedding
Absolute Position Embedding
RoPE Theta
1,000,000
Normalization
RMS Normalization
Activation Function
SwigLU
Dimensions
Base Precision
FP16
Hidden Dimension Size
12,288
Number of Layers
88
FFN Intermediate Size (Dense)
28,672
Tokenizer
Vocabulary Size
32,768
Mixture of Experts
Total Expert Parameters
675.0B
Number of Experts
16
Active Experts
2
About Mistral Large 3
Mistral Large 3 is a state-of-the-art general-purpose multimodal model with a granular Mixture-of-Experts architecture. With 675B total parameters and 41B active parameters, it delivers frontier performance for production-grade assistants, retrieval-augmented systems, and complex enterprise workflows.
Other Mistral Large 3 Models
- No related models available