Parameters
123B
Context Length
128K
Modality
Text
Architecture
Dense
KV Cache Rate
512 KB/tok
License
Mistral Research License
Release Date
24 Jul 2024
Knowledge Cutoff
Mar 2024
API Pricing (per 1M)
Input: $2.00 · Output: $6.00
Precision
Context
Workload Memory: 235.61 GB
FP16 · 8K context
Base Weights: 229.11 GB
KV Cache: 4.00 GB
Runtime & Graphs: 2.50 GB
8x NVIDIA H100 80GB
235.6 / 576.0 GB usable · TP 8
Determine requirements for your specific workload:
Open in VRAM Calculator
Checkpoint format
FP16
CUDA
>= 11.8
PyTorch
>= 2.1
Transformers
>= 4.40.1
| Architecture | Precision | Attention kernel |
|---|---|---|
Blackwell, Hopper | Native FP16 | FlashAttention-3 |
Ada Lovelace, Ampere | Native FP16 | FlashAttention-2 |
Rank
#186
| Benchmark | Score | Rank |
|---|---|---|
General Text | 1315 | 164 |
Intelligence Index | 0.08 | 273 |
QA Assistant Archived | 0.964 | 5 |
StackEval Archived | 0.876 | 10 |
General Knowledge Reference | 0.84 | 14 |
Summarization Archived | 0.729 | 19 |
Overall Rank
#186
Coding Rank
-
Mistral Large 2 (2407) is Mistral AI's flagship 123B dense transformer model optimized for single-node enterprise inference and advanced code generation. It supports over 80 programming languages and precise function calling across a 128K context window.
Attention
Attention Structure
Grouped-Query Attention
Attention Heads
48
Key-Value Heads
8
Position Embedding
ROPE
Normalization
RMS Normalization
Activation Function
SwigLU
Dimensions
Base Precision
FP16
Hidden Dimension Size
12,288
Number of Layers
64
Mistral Large 2 is a 123 billion parameter, dense transformer model engineered for advanced language and code generation, supporting over 80 programming languages. Its 128,000 token context window facilitates complex reasoning and long-context applications on a single node. Enhanced function calling capabilities are integrated.