Parameters
8B
Context Length
131K
Modality
Text
Architecture
Dense
KV Cache Rate
128 KB/tok
License
Llama 3.1 Community License
Release Date
23 Jul 2024
Knowledge Cutoff
Dec 2023
API Pricing (per 1M)
Input: $0.02 · Output: $0.04
Precision
Context
Workload Memory: 7.23 GB
INT4 · 8K context
Base Weights: 3.73 GB
KV Cache: 1.00 GB
Runtime & Graphs: 2.50 GB
1x NVIDIA L4 24GB
7.2 / 21.6 GB usable
1x RTX 3060 12GB
7.2 / 10.8 GB usable
Apple M4 24GB
7.2 / 18.0 GB usable
Determine requirements for your specific workload:
Open in VRAM Calculator
Checkpoint format
FP16
CUDA
>= 11.8
PyTorch
>= 2.1
Transformers
>= 4.40.1
| Architecture | Precision | Attention kernel |
|---|---|---|
Blackwell, Hopper | Native FP16 | FlashAttention-3 |
Ada Lovelace, Ampere | Native FP16 | FlashAttention-2 |
Rank
#224
| Benchmark | Score | Rank |
|---|---|---|
Professional Knowledge | 0.483 | 58 |
Graduate-Level QA | 0.304 | 121 |
Coding Index | 0.05 | 153 |
General Text | 1211.28 | 191 |
Intelligence Index | 0.07 | 295 |
StackEval Archived | 0.502 | 17 |
Summarization Archived | 0.491 | 27 |
General Knowledge Reference | 0.694 | 27 |
Overall Rank
#224
Coding Rank
#170
Llama 3.1 8B is an efficient open foundation model by Meta engineered for multilingual dialogue, classification, and edge deployment across 8 languages. It features an expanded 128K token context window and enhanced tool-calling capabilities.
Attention
Attention Structure
Grouped-Query Attention
Attention Heads
32
Key-Value Heads
8
Position Embedding
ROPE
Normalization
RMS Normalization
Dimensions
Base Precision
FP16
Hidden Dimension Size
4,096
Number of Layers
32
Llama 3.1 is Meta's advanced large language model family, building upon Llama 3. It features an optimized decoder-only transformer architecture, available in 8B, 70B, and 405B parameter versions. Significant enhancements include an expanded 128K token context window and improved multilingual capabilities across eight languages, refined through data and post-training procedures.