Gemma 3 1B

Parameters

1B

Context Length

33K

Modality

Text

Architecture

Dense

KV Cache Rate

39 KB/tok

License

Gemma License

Release Date

12 Mar 2025

Knowledge Cutoff

Aug 2024

API Pricing (per 1M)

Input: $0.00 · Output: $0.00

Hardware & System Requirements

Precision

Native

Context

Workload Memory: 3.27 GB

INT4 · 8K context

Base Weights: 0.47 GB

KV Cache: 0.30 GB

Runtime & Graphs: 2.50 GB

Minimum Required Hardware

1x NVIDIA L4 24GB

3.3 / 21.6 GB usable

1x RTX 3060 12GB

3.3 / 10.8 GB usable

Apple M4 24GB

3.3 / 18.0 GB usable

Determine requirements for your specific workload:

Open in VRAM Calculator

Runtime requirements

Checkpoint format

FP16

CUDA

>= 11.8

PyTorch

>= 2.1

Transformers

>= 4.40.1

ArchitecturePrecisionAttention kernel

Blackwell, Hopper

Native FP16

FlashAttention-3

Ada Lovelace, Ampere

Native FP16

FlashAttention-2

Architecture Diagram

Input TokensToken EmbeddingPosition: RoPEHidden: 1.5k · Context: 33Kx 26 layersRMSNormPre-AttentionGrouped-Query Attention16Q / 4KV headsHead dim: 96+RMSNormPre-FFNFeed-Forward NetworkActivation+Final RMSNormOutput Logits

Evaluation Benchmarks

Rank

#247

BenchmarkScoreRank

Professional Knowledge

MMLU Pro

0.147

63

Graduate-Level QA

GPQA

0.192

125

Intelligence Index

Artificial Analysis

0.05

342

Rankings

Overall Rank

#247

Coding Rank

-

About Gemma 3 1B

Gemma 3 1B is Google's compact, text-only open model engineered for high-throughput, low-latency execution on mobile and edge devices. It delivers fast local generation and multilingual comprehension across a 32K token context window.

Specifications

Attention

Attention Structure

Grouped-Query Attention

Attention Heads

16

Key-Value Heads

4

Position Embedding

ROPE

Normalization

RMS Normalization

Dimensions

Base Precision

FP16

Hidden Dimension Size

1,536

Number of Layers

26

About Gemma 3

Gemma 3 is a family of open, lightweight models from Google. It introduces multimodal image and text processing, supports over 140 languages, and features extended context windows up to 128K tokens. Models are available in multiple parameter sizes for diverse applications.


Other Gemma 3 Models