Llama 3 70B
Parameters
70B
Context Length
8K
Modality
Text
Architecture
Dense
KV Cache Rate
320 KB/tok
License
Meta Llama 3 Community License
Release Date
18 Apr 2024
Knowledge Cutoff
Dec 2023
API Pricing (per 1M)
Input: $0.51 · Output: $0.74
Hardware Sizing
Precision
Context
Workload Memory: 37.60 GB
INT4 · 8K context
Base Weights: 32.60 GB
KV Cache: 2.50 GB
Runtime & Graphs: 2.50 GB
Minimum Required Hardware
1x NVIDIA A40 48GB
37.6 / 43.2 GB usable
2x RTX 4090 24GB
37.6 / 43.2 GB usable · TP 2
Apple M4 Max 128GB
37.6 / 96.0 GB usable
Determine requirements for your specific workload:
Open in VRAM Calculator
Architecture Diagram
Evaluation Benchmarks
Rank
#197
| Benchmark | Score | Rank |
|---|---|---|
General Text | 1277 | 172 |
Intelligence Index | 0.06 | 301 |
Rankings
Overall Rank
#197
Coding Rank
-
About Llama 3 70B
Llama 3 70B is a high-capacity open-weights model developed by Meta for complex reasoning, long-form summarization, and conversational AI. It is instruction-tuned for helpfulness and safety across enterprise and research environments.
Technical Specifications
Attention
Attention Structure
Grouped-Query Attention
Attention Heads
64
Key-Value Heads
8
Position Embedding
ROPE
Dimensions
Hidden Dimension Size
8,192
Number of Layers
80
About Llama 3
Meta's Llama 3 is a series of large language models utilizing a decoder-only transformer architecture. It incorporates a 128K token vocabulary and Grouped Query Attention for efficient processing. Models are trained on substantial public datasets, supporting various parameter scales and extended context lengths.