Llama 3 8B
Parameters
8B
Context Length
8K
Modality
Text
Architecture
Dense
KV Cache Rate
128 KB/tok
License
Meta Llama 3 Community License Agreement
Release Date
18 Apr 2024
Knowledge Cutoff
Mar 2023
API Pricing (per 1M)
Input: $0.14 · Output: $0.14
Hardware & System Requirements
Precision
Context
Workload Memory: 7.23 GB
INT4 · 8K context
Base Weights: 3.73 GB
KV Cache: 1.00 GB
Runtime & Graphs: 2.50 GB
Minimum Required Hardware
1x NVIDIA L4 24GB
7.2 / 21.6 GB usable
1x RTX 3060 12GB
7.2 / 10.8 GB usable
Apple M4 24GB
7.2 / 18.0 GB usable
Determine requirements for your specific workload:
Open in VRAM Calculator
Runtime requirements
Checkpoint format
FP16
CUDA
>= 11.8
PyTorch
>= 2.1
Transformers
>= 4.40.1
| Architecture | Precision | Attention kernel |
|---|---|---|
Blackwell, Hopper | Native FP16 | FlashAttention-3 |
Ada Lovelace, Ampere | Native FP16 | FlashAttention-2 |
Architecture Diagram
Evaluation Benchmarks
Rank
#220
| Benchmark | Score | Rank |
|---|---|---|
General Text | 1224 | 172 |
Intelligence Index | 0.05 | 323 |
Rankings
Overall Rank
#220
Coding Rank
-
About Llama 3 8B
Llama 3 8B is Meta's foundational open-weights language model pretrained on over 15T tokens for text generation, coding, and dialogue tasks. It features Grouped-Query Attention and an optimized 128K vocabulary for efficient inference.
Specifications
Attention
Attention Structure
Grouped-Query Attention
Attention Heads
32
Key-Value Heads
8
Position Embedding
ROPE
Normalization
RMS Normalization
Activation Function
SwigLU
Dimensions
Base Precision
FP16
Hidden Dimension Size
4,096
Number of Layers
32
About Llama 3
Meta's Llama 3 is a series of large language models utilizing a decoder-only transformer architecture. It incorporates a 128K token vocabulary and Grouped Query Attention for efficient processing. Models are trained on substantial public datasets, supporting various parameter scales and extended context lengths.