Parameters
2B
Context Length
8K
Modality
Text
Architecture
Dense
KV Cache Rate
48 KB/tok
License
Apache 2.0
Release Date
14 Sept 2026
Knowledge Cutoff
-
API Pricing (per 1M)
Self-hosted only
Precision
Workload Memory: 3.43 GB
INT4
Base Weights: 0.93 GB
Runtime & Graphs: 2.50 GB
1x NVIDIA L4 24GB
3.4 / 21.6 GB usable
1x RTX 3060 12GB
3.4 / 10.8 GB usable
Apple M4 24GB
3.4 / 18.0 GB usable
Determine requirements for your specific workload:
Open in VRAM Calculator
Checkpoint format
FP16
CUDA
>= 11.8
PyTorch
>= 2.1
Transformers
>= 5.18.0.dev0
| Architecture | Precision | Attention kernel |
|---|---|---|
Blackwell, Hopper | Native FP16 | FlashAttention-3 |
Ada Lovelace, Ampere | Native FP16 | FlashAttention-2 |
No evaluation benchmarks for EmbeddingGemma 2 available.
Overall Rank
-
Coding Rank
-
EmbeddingGemma 2 is a specialized text embedding model based on Google's Gemma 2 architecture. It is designed for high-performance semantic search, retrieval-augmented generation, and text similarity tasks.
Attention
Attention Structure
Single-Head Attention
Attention Heads
4
Key-Value Heads
2
Attention Head Dimension
256
Position Embedding
ROPE
RoPE Theta
10,000
Sliding Window Attention
Yes
Sliding Window Size
512
Sliding Window Ratio
83.3%
Normalization
RMS Normalization
Activation Function
Gated GELU
Dimensions
Base Precision
FP16
Hidden Dimension Size
512
Number of Layers
24
FFN Intermediate Size (Dense)
2,048
Tokenizer
Vocabulary Size
262,144
Gemma 2 is Google's family of open large language models, offering 2B, 9B, and 27B parameter sizes. Built upon the Gemma architecture, it incorporates innovations such as interleaved local and global attention, logit soft-capping for training stability, and Grouped Query Attention for inference efficiency. The smaller models leverage knowledge distillation.