ApX logoApX logo

EmbeddingGemma 2

Parameters

2B

Context Length

8K

Modality

Text

Architecture

Dense

KV Cache Rate

48 KB/tok

License

Apache 2.0

Release Date

14 Sept 2026

Knowledge Cutoff

-

API Pricing (per 1M)

Self-hosted only

Hardware & System Requirements

Precision

Native

Workload Memory: 3.43 GB

INT4

Base Weights: 0.93 GB

Runtime & Graphs: 2.50 GB

Minimum Required Hardware

1x NVIDIA L4 24GB

3.4 / 21.6 GB usable

1x RTX 3060 12GB

3.4 / 10.8 GB usable

Apple M4 24GB

3.4 / 18.0 GB usable

Determine requirements for your specific workload:

Open in VRAM Calculator

Runtime requirements

Checkpoint format

FP16

CUDA

>= 11.8

PyTorch

>= 2.1

Transformers

>= 5.18.0.dev0

ArchitecturePrecisionAttention kernel

Blackwell, Hopper

Native FP16

FlashAttention-3

Ada Lovelace, Ampere

Native FP16

FlashAttention-2

Architecture Diagram

Evaluation Benchmarks

No evaluation benchmarks for EmbeddingGemma 2 available.

Rankings

Overall Rank

-

Coding Rank

-

About EmbeddingGemma 2

EmbeddingGemma 2 is a specialized text embedding model based on Google's Gemma 2 architecture. It is designed for high-performance semantic search, retrieval-augmented generation, and text similarity tasks.

Specifications

Attention

Attention Structure

Single-Head Attention

Attention Heads

4

Key-Value Heads

2

Attention Head Dimension

256

Position Embedding

ROPE

RoPE Theta

10,000

Sliding Window Attention

Yes

Sliding Window Size

512

Sliding Window Ratio

83.3%

Normalization

RMS Normalization

Activation Function

Gated GELU

Dimensions

Base Precision

FP16

Hidden Dimension Size

512

Number of Layers

24

FFN Intermediate Size (Dense)

2,048

Tokenizer

Vocabulary Size

262,144

About Gemma 2

Gemma 2 is Google's family of open large language models, offering 2B, 9B, and 27B parameter sizes. Built upon the Gemma architecture, it incorporates innovations such as interleaved local and global attention, logit soft-capping for training stability, and Grouped Query Attention for inference efficiency. The smaller models leverage knowledge distillation.


Other Gemma 2 Models