Llama 3.1 8B

Parameters

8B

Context Length

131K

Modality

Text

Architecture

Dense

KV Cache Rate

128 KB/tok

License

Llama 3.1 Community License

Release Date

23 Jul 2024

Knowledge Cutoff

Dec 2023

API Pricing (per 1M)

Input: $0.02 · Output: $0.04

Hardware & System Requirements

Precision

Native

Context

Workload Memory: 7.23 GB

INT4 · 8K context

Base Weights: 3.73 GB

KV Cache: 1.00 GB

Runtime & Graphs: 2.50 GB

Minimum Required Hardware

1x NVIDIA L4 24GB

7.2 / 21.6 GB usable

1x RTX 3060 12GB

7.2 / 10.8 GB usable

Apple M4 24GB

7.2 / 18.0 GB usable

Determine requirements for your specific workload:

Open in VRAM Calculator

Runtime requirements

Checkpoint format

FP16

CUDA

>= 11.8

PyTorch

>= 2.1

Transformers

>= 4.40.1

ArchitecturePrecisionAttention kernel

Blackwell, Hopper

Native FP16

FlashAttention-3

Ada Lovelace, Ampere

Native FP16

FlashAttention-2

Architecture Diagram

Evaluation Benchmarks

Rank

#224

BenchmarkScoreRank

Professional Knowledge

MMLU Pro

0.483

58

Graduate-Level QA

GPQA

0.304

121

0.05

153

General Text

Text Arena

1211.28

191

Intelligence Index

Artificial Analysis

0.07

295

StackEval

Archived
ProLLM Stack Eval

0.502

17

Summarization

Archived
ProLLM Summarization

0.491

27

General Knowledge

Reference
MMLU

0.694

27

Rankings

Overall Rank

#224

Coding Rank

#170

About Llama 3.1 8B

Llama 3.1 8B is an efficient open foundation model by Meta engineered for multilingual dialogue, classification, and edge deployment across 8 languages. It features an expanded 128K token context window and enhanced tool-calling capabilities.

Specifications

Attention

Attention Structure

Grouped-Query Attention

Attention Heads

32

Key-Value Heads

8

Position Embedding

ROPE

Normalization

RMS Normalization

Dimensions

Base Precision

FP16

Hidden Dimension Size

4,096

Number of Layers

32

About Llama 3.1

Llama 3.1 is Meta's advanced large language model family, building upon Llama 3. It features an optimized decoder-only transformer architecture, available in 8B, 70B, and 405B parameter versions. Significant enhancements include an expanded 128K token context window and improved multilingual capabilities across eight languages, refined through data and post-training procedures.


Other Llama 3.1 Models