Gemma 3 27B
Parameters
27B
Context Length
128K
Modality
Multimodal
Auxiliary Parameters
430M
Architecture
Dense
KV Cache Rate
184 KB/tok
License
Gemma Terms of Use
Release Date
12 Mar 2025
Knowledge Cutoff
Aug 2024
API Pricing (per 1M)
Input: $0.08 · Output: $0.35
Hardware & System Requirements
Precision
Context
Workload Memory: 16.51 GB
INT4 · 8K context
Base Weights: 12.57 GB
KV Cache: 1.44 GB
Runtime & Graphs: 2.50 GB
Minimum Required Hardware
1x NVIDIA L4 24GB
16.5 / 21.6 GB usable
2x RTX 4090 24GB
16.5 / 43.2 GB usable · TP 2
Apple M4 24GB
16.5 / 18.0 GB usable
Determine requirements for your specific workload:
Open in VRAM Calculator
Runtime requirements
Checkpoint format
FP16
CUDA
>= 11.8
PyTorch
>= 2.1
Transformers
>= 4.40.1
| Architecture | Precision | Attention kernel |
|---|---|---|
Blackwell, Hopper | Native FP16 | FlashAttention-3 |
Ada Lovelace, Ampere | Native FP16 | FlashAttention-2 |
Architecture Diagram
Evaluation Benchmarks
Rank
#167
| Benchmark | Score | Rank |
|---|---|---|
StackUnseen | 0.372 | 28 |
Professional Knowledge | 0.675 | 48 |
Graduate-Level QA | 0.424 | 113 |
Agentic Index | 0.1 | 127 |
General Text | 1365 | 143 |
Coding Index | 0.10 | 146 |
Intelligence Index | 0.05 | 321 |
StackEval Archived | 0.905 | 8 |
Summarization Archived | 0.802 | 12 |
QA Assistant Archived | 0.913 | 14 |
Coding Archived | 0.05 | 24 |
Rankings
Overall Rank
#167
Coding Rank
#136
About Gemma 3 27B
Gemma 3 is a family of lightweight, state-of-the-art models developed by Google DeepMind, designed with research and technology derived from the Gemini models. The Gemma 3 27B variant is a multimodal model engineered to process both textual and image inputs, generating text-based outputs. This model variant is intended for broad application across various generation tasks, including question answering, summarization, and complex reasoning, and supports over 140 languages. Its design focuses on enabling deployment on a range of hardware, from consumer-grade devices like laptops and workstations to specialized cloud infrastructure.
Specifications
Attention
Attention Structure
Grouped-Query Attention
Attention Heads
64
Key-Value Heads
16
Position Embedding
ROPE
Normalization
RMS Normalization
Dimensions
Base Precision
FP16
Auxiliary Parameters
430M
Hidden Dimension Size
4,096
Number of Layers
46
About Gemma 3
Gemma 3 is a family of open, lightweight models from Google. It introduces multimodal image and text processing, supports over 140 languages, and features extended context windows up to 128K tokens. Models are available in multiple parameter sizes for diverse applications.