ApX logoApX logo

Llama 3.2 1B

Parameters

1B

Context Length

128K

Modality

Text

Architecture

Dense

KV Cache Rate

16 KB/tok

License

Llama 3.2 Community License

Release Date

25 Sept 2024

Knowledge Cutoff

Dec 2023

API Pricing (per 1M)

Input: $0.03 · Output: $0.20

Hardware & System Requirements

Precision

Native

Context

Workload Memory: 3.09 GB

INT4 · 8K context

Base Weights: 0.47 GB

KV Cache: 0.13 GB

Runtime & Graphs: 2.50 GB

Minimum Required Hardware

1x NVIDIA L4 24GB

3.1 / 21.6 GB usable

1x RTX 3060 12GB

3.1 / 10.8 GB usable

Apple M4 24GB

3.1 / 18.0 GB usable

Determine requirements for your specific workload:

Open in VRAM Calculator

Runtime requirements

Checkpoint format

FP16

CUDA

>= 11.8

PyTorch

>= 2.1

Transformers

>= 4.52.0.dev0

ArchitecturePrecisionAttention kernel

Blackwell, Hopper

Native FP16

FlashAttention-3

Ada Lovelace, Ampere

Native FP16

FlashAttention-2

Architecture Diagram

Input TokensToken EmbeddingPosition: RoPEHidden: 1k · Context: 128K · Vocab: 128.3kx 16 layersRMSNormPre-AttentionGrouped-Query Attention16Q / 4KV headsHead dim: 64+RMSNormPre-FFNFeed-Forward NetworkSwiGLUIntermediate: 8.2k+Final RMSNormOutput Logits

Evaluation Benchmarks

Rank

#235

BenchmarkScoreRank

General Text

Text Arena

1111

195

Intelligence Index

Artificial Analysis

0.05

330

Rankings

Overall Rank

#235

Coding Rank

-

About Llama 3.2 1B

Llama 3.2 1B is an ultra-compact edge model by Meta optimized for on-device summarization, rewriting, and multilingual retrieval. Distilled from larger Llama 3.1 models, it supports local inference across a 128K token context window.

Specifications

Attention

Attention Structure

Grouped-Query Attention

Attention Heads

16

Key-Value Heads

4

Attention Head Dimension

64

Position Embedding

ROPE

RoPE Theta

500,000

Normalization

RMS Normalization

Activation Function

SwigLU

Dimensions

Base Precision

FP16

Hidden Dimension Size

1,024

Number of Layers

16

FFN Intermediate Size (Dense)

8,192

Tokenizer

Vocabulary Size

128,256

About Llama 3.2

Meta's Llama 3.2 family introduces vision models, integrating image encoders with language models for multimodal text and image processing. It also includes lightweight variants optimized for efficient on-device deployment, supporting an extended 128K token context length.


Other Llama 3.2 Models