Gemma 4 E2B

Gemma 4 E2B is explicitly documented as a decoder-only Transformer with several specific modifications: Per-Layer Embeddings (PLE), Alternating Attention (sliding-window vs. global), and Shared KV Cache. The model card and technical blog posts detail the layer count (35) and the specific 4:1 ratio of local to global attention layers. While the training methodology (knowledge distillation from Gemini 3) is mentioned, the exact pre-training procedure and data mixtures remain high-level rather than fully reproducible.

Information regarding the training data is extremely limited. Official documentation states the model was trained on a 'diverse collection of datasets' including web text, code, and multimodal data (images/audio), but provides no specific percentage breakdowns, source names, or detailed filtering/cleaning methodologies. The lack of data provenance is a significant transparency gap common to the Gemma family.

The tokenizer is publicly available via the 'gemma' GitHub repository and Hugging Face. It uses a SentencePiece-based approach with a large vocabulary of 262,144 tokens, supporting over 140 languages. Documentation clearly specifies the tokenization of multimodal inputs (e.g., 16x16 patches for images and mel-spectrograms for audio) and how they are interleaved into the text sequence.

Google provides clear distinctions between 'effective' and 'total' parameters. The E2B variant has 2.3B active parameters during inference but 5.1B total parameters due to the Per-Layer Embeddings (PLE) architecture. This breakdown is consistently reported across official model cards and third-party technical reviews, clarifying the memory vs. compute trade-offs.

There is almost no public disclosure regarding the specific compute resources used to train Gemma 4. While it is known to be trained on Google's TPU infrastructure, the total TPU/GPU hours, carbon footprint, and specific hardware counts are not provided in the release documentation or model cards.

While Google provides scores for standard benchmarks (MMLU Pro, AIME 2026, LiveCodeBench), the exact evaluation prompts and few-shot examples are not fully disclosed in a reproducible harness. Third-party verification from platforms like Artificial Analysis and LMSys Chatbot Arena exists, but the internal 'thinking mode' benchmarks lack public validation scripts.

The model demonstrates high identity consistency, correctly identifying itself as a Google-developed model within the Gemma 4 family. It is aware of its versioning and its specific multimodal capabilities (text, image, audio). There are no documented cases of the model claiming to be a competitor's product or misrepresenting its underlying architecture.

Gemma 4 marks a significant shift to the Apache 2.0 license, which is a standard, permissive open-source license. The terms are unambiguous, allowing for unrestricted commercial use, modification, and distribution. This is a major improvement over previous custom 'Gemma Terms of Use' and provides maximum legal clarity for developers.

Hardware requirements are well-documented for various deployment scenarios. Documentation specifies that the E2B variant can run in under 1.5GB of RAM with 4-bit quantization and provides specific VRAM targets for FP16 (10GB) and 8-bit (5-8GB). Performance metrics for edge devices like Raspberry Pi 5 and mobile NPUs are also publicly available.

The model uses clear naming conventions (E2B, E4B, IT variants), but a formal semantic versioning changelog for weight updates is not yet established. While the initial release is well-documented, there is no public framework for tracking silent updates or performance drift over time beyond the initial model card.