Gemma 4 31B

Gemma 4 31B is explicitly documented as a dense Transformer-based model built upon the research foundations of Gemini 3. Technical documentation specifies a hybrid attention mechanism that interleaves local sliding window attention (1024 token window for this variant) with global attention layers. It incorporates Proportional RoPE (p-RoPE) and unified Keys and Values in global layers to manage its 256K context window. While the high-level architecture is well-described, the specific pre-training recipe and the exact interleaving pattern of attention layers are not fully disclosed in the public model card.

Google provides only high-level thematic descriptions of the training data, citing 'large-scale multimodal pre-training data' including web documents, code, images, and audio with a cutoff of January 2025. There is no specific percentage breakdown of dataset components (e.g., what portion is code vs. web text), no disclosure of specific data sources, and no detailed documentation of the filtering or cleaning pipeline beyond vague mentions of 'quality and safety' filters. This lack of granularity makes the dataset composition largely unverifiable.

The tokenizer is publicly accessible via the 'transformers' library and official GitHub repositories. It uses a Byte Pair Encoding (BPE) approach with a vocabulary size of 256,144 tokens, supporting over 140 languages. Documentation clearly states the use of byte fallback and specific normalization (replacing spaces with a specific character). The vocabulary size and behavior are consistent across API and local implementations, though the exact training data for the tokenizer itself is not detailed.

The model's parameter density is clearly stated as 30.7B total parameters. As a dense model, all parameters are active during inference, which is explicitly clarified in contrast to the MoE (26B A4B) variant in the same family. Architectural details such as the number of layers (60) and the wider model dimension compared to previous generations are provided in technical overviews, offering a high degree of transparency regarding its physical structure.

There is almost no transparency regarding the compute resources used to train Gemma 4 31B. While it is known to be trained on Google's TPU infrastructure (v4/v5), the specific number of TPU/GPU hours, the total energy consumption, and the carbon footprint are not disclosed in the official model card or technical documentation. This information is withheld for competitive reasons, leaving the environmental and resource impact entirely opaque.

Google provides extensive benchmark results (AIME 2026, MMLU Pro, LiveCodeBench v6) and third-party verification is available through the LMSYS Chatbot Arena (where it ranks #3 among open models). However, the exact evaluation code, specific few-shot prompts, and detailed reproduction instructions are not fully public. While the results are verifiable by third parties, the lack of a standardized, public evaluation harness for the specific reported scores limits full reproducibility.

The model demonstrates high identity consistency, correctly identifying itself as a Google-developed model within the Gemma 4 family. It is transparent about its versioning (31B IT) and its multimodal capabilities (text and image input). There are no documented cases of the model claiming to be a competitor's product or misrepresenting its core architectural nature as a dense model.

Gemma 4 31B is released under the standard Apache 2.0 license, a significant and transparent shift from the custom 'Gemma Terms of Use' seen in previous versions. This license is globally recognized, permits unrestricted commercial use, modification, and redistribution, and contains no hidden usage restrictions or 'trust me' clauses. The legal terms are clear, public, and standard across the industry.

Hardware requirements are well-documented by both Google and third-party maintainers (e.g., Unsloth, LM Studio). VRAM requirements are specified for various precisions: ~62GB for BF16, ~34GB for Q8, and ~20GB for Q4. Documentation also includes memory scaling data for the 256K context window, noting that a 24GB GPU can handle up to ~45K tokens. The trade-offs between quantization levels and performance are widely discussed in the community and supported by official guidance.

The model uses clear naming conventions (Gemma 4 31B IT) and weights are hosted on Hugging Face with basic commit histories. However, Google does not provide a detailed public changelog or a formal policy on model drift and silent updates. While versioning exists at the release level, the long-term commitment to maintaining access to specific 'frozen' versions versus rolling updates is not explicitly documented.