ApX logoApX logo

Mistral-Large-2407

Parameters

123B

Context Length

128K

Modality

Text

Architecture

Dense

KV Cache Rate

512 KB/tok

License

Mistral Research License

Release Date

24 Jul 2024

Knowledge Cutoff

Mar 2024

API Pricing (per 1M)

Input: $2.00 · Output: $6.00

Hardware & System Requirements

Precision

Native

Context

Workload Memory: 235.61 GB

FP16 · 8K context

Base Weights: 229.11 GB

KV Cache: 4.00 GB

Runtime & Graphs: 2.50 GB

Minimum Required Hardware

8x NVIDIA H100 80GB

235.6 / 576.0 GB usable · TP 8

Determine requirements for your specific workload:

Open in VRAM Calculator

Runtime requirements

Checkpoint format

FP16

CUDA

>= 11.8

PyTorch

>= 2.1

Transformers

>= 4.40.1

ArchitecturePrecisionAttention kernel

Blackwell, Hopper

Native FP16

FlashAttention-3

Ada Lovelace, Ampere

Native FP16

FlashAttention-2

Architecture Diagram

Input TokensToken EmbeddingPosition: RoPEHidden: 12.3k · Context: 128Kx 64 layersRMSNormPre-AttentionGrouped-Query Attention48Q / 8KV headsHead dim: 256+RMSNormPre-FFNFeed-Forward NetworkSwiGLU+Final RMSNormOutput Logits

Evaluation Benchmarks

Rank

#186

BenchmarkScoreRank

General Text

Text Arena

1315

164

Intelligence Index

Artificial Analysis

0.08

273

QA Assistant

Archived
ProLLM QA Assistant

0.964

5

StackEval

Archived
ProLLM Stack Eval

0.876

10

General Knowledge

Reference
MMLU

0.84

14

Summarization

Archived
ProLLM Summarization

0.729

19

Rankings

Overall Rank

#186

Coding Rank

-

About Mistral-Large-2407

Mistral Large 2 (2407) is Mistral AI's flagship 123B dense transformer model optimized for single-node enterprise inference and advanced code generation. It supports over 80 programming languages and precise function calling across a 128K context window.

Specifications

Attention

Attention Structure

Grouped-Query Attention

Attention Heads

48

Key-Value Heads

8

Position Embedding

ROPE

Normalization

RMS Normalization

Activation Function

SwigLU

Dimensions

Base Precision

FP16

Hidden Dimension Size

12,288

Number of Layers

64

About Mistral Large 2

Mistral Large 2 is a 123 billion parameter, dense transformer model engineered for advanced language and code generation, supporting over 80 programming languages. Its 128,000 token context window facilitates complex reasoning and long-context applications on a single node. Enhanced function calling capabilities are integrated.


Other Mistral Large 2 Models
  • No related models available