Gemma 4 26B A4B
Active Parameters
25.2B
Context Length
256K
Modality
Multimodal
Auxiliary Parameters
450M
Architecture
Mixture of Experts
KV Cache Rate
240 KB/tok
License
Apache 2.0
Release Date
2 Apr 2026
Knowledge Cutoff
-
API Pricing (per 1M)
Input: $0.05 · Output: $0.29
Hardware Sizing
Precision
Context
Workload Memory: 17.88 GB
INT4 · 8K context
Base Weights: 13.50 GB
KV Cache: 1.88 GB
Runtime & Graphs: 2.50 GB
Minimum Required Hardware
1x NVIDIA L4 24GB
17.9 / 21.6 GB usable
2x RTX 4090 24GB
17.9 / 43.2 GB usable · TP 2
Apple M4 24GB
17.9 / 18.0 GB usable
Determine requirements for your specific workload:
Open in VRAM Calculator
Architecture Diagram
Evaluation Benchmarks
Rank
#117
| Benchmark | Score | Rank |
|---|---|---|
Professional Knowledge | 0.826 | 20 |
Graduate-Level QA | 0.823 | 57 |
General Text | 1437 | 81 |
Web Development | 1358 | 98 |
Agentic Index | auto 0.03 | 105 |
Coding Index | 0.39 | 107 |
Intelligence Index | auto 0.17 Standard 0.13 | 197 217 |
Rankings
Overall Rank
#117
Coding Rank
#112
About Gemma 4 26B A4B
Gemma 4 26B A4B is a Mixture-of-Experts model with 25.2B total parameters but only 3.8B active per inference, achieving the speed of a 4B model with near-31B performance. Features 128 experts (8 active) with 256K context window, supporting text and image input. Optimized for fast inference on consumer GPUs while delivering frontier-level reasoning and coding capabilities.
Technical Specifications
Attention
Attention Structure
Grouped-Query Attention
Attention Heads
16
Key-Value Heads
8
Attention Head Dimension
256
Position Embedding
ROPE
RoPE Theta
10,000
Sliding Window Attention
Yes
Sliding Window Size
1,024
Sliding Window Ratio
83.3%
Normalization
RMS Normalization
Activation Function
GELU
Dimensions
Auxiliary Parameters
450M
Hidden Dimension Size
2,112
Number of Layers
30
FFN Intermediate Size (Dense)
704
Tokenizer
Vocabulary Size
262,144
Mixture of Experts
Total Expert Parameters
3.8B
Number of Experts
128
Active Experts
8
FFN Intermediate Size (per Expert)
704
About Gemma 4
Gemma 4 is Google DeepMind's most advanced open model family, built from Gemini 3 research and technology. Featuring both Dense and Mixture-of-Experts (MoE) architectures, these multimodal models handle text, images, and audio (on smaller variants), with context windows up to 256K tokens. Designed for frontier-level performance across reasoning, coding, and agentic workflows, Gemma 4 delivers unprecedented intelligence-per-parameter from mobile devices to enterprise servers. Released under Apache 2.0 license.