Gemma 4 31B
Parameters
30.7B
Context Length
256K
Modality
Multimodal
Auxiliary Parameters
450M
Architecture
Dense
License
Apache 2.0
Release Date
2 Apr 2026
Knowledge Cutoff
-
API Pricing (per 1M)
Input: $0.00 · Output: $0.00
System Requirements
VRAM requirements for different quantization methods and context sizes
1,024 tokens
Consumer
3x RTX 4090
24GB VRAM
Datacenter
1x NVIDIA A100
80GB VRAM
Apple Silicon
1x Apple M3 Max
128GB VRAM
256,000 tokens
Consumer
16x RTX 4090
24GB VRAM
Datacenter
5x NVIDIA A100
80GB VRAM
Apple Silicon
3x Apple M3 Max
128GB VRAM
Architecture Diagram
Evaluation Benchmarks
Rank
#131
| Benchmark | Score | Rank |
|---|---|---|
Professional Knowledge | 0.852 | 10 |
Graduate-Level QA | 0.843 | 47 |
Agent Arena | -23.7 | 58 |
General Text | 1453 | 62 |
Web Development | 1363 | 83 |
Agentic Index | 0.04 | 110 |
Coding Index | 0.33 | 119 |
Intelligence Index | 0.14 | 189 |
Rankings
Overall Rank
#131
Coding Rank
#123
About Gemma 4 31B
Gemma 4 31B is the flagship dense model with 30.7B parameters and 256K context window, delivering frontier intelligence for workstations and consumer GPUs. Supports text and image input with state-of-the-art performance on coding, reasoning, and multimodal understanding. Features configurable thinking mode and native function calling for advanced agentic workflows and IDE integration.
Technical Specifications
Attention
Attention Structure
Grouped-Query Attention
Attention Heads
32
Key-Value Heads
16
Attention Head Dimension
256
Position Embedding
ROPE
RoPE Theta
1,000,000
Sliding Window Attention
Yes
Sliding Window Size
1,024
Sliding Window Ratio
83.3%
Linear Attention
No
Linear Attention Ratio
-
Normalization
RMS Normalization
Activation Function
GELU
Dimensions
Auxiliary Parameters
450M
Hidden Dimension Size
5,376
Number of Layers
60
FFN Intermediate Size (Dense)
21,504
Multi-Token Prediction Heads
-
Tokenizer
Vocabulary Size
262,144
About Gemma 4
Gemma 4 is Google DeepMind's most advanced open model family, built from Gemini 3 research and technology. Featuring both Dense and Mixture-of-Experts (MoE) architectures, these multimodal models handle text, images, and audio (on smaller variants), with context windows up to 256K tokens. Designed for frontier-level performance across reasoning, coding, and agentic workflows, Gemma 4 delivers unprecedented intelligence-per-parameter from mobile devices to enterprise servers. Released under Apache 2.0 license.