Gemma 3n E2B IT

The model's architecture is extensively documented as a 'Matryoshka Transformer' (MatFormer) with specific innovations like Per-Layer Embeddings (PLE) and selective parameter activation. Technical documentation and the Gemma 3 Technical Report (arXiv:2503.19786) detail the 5:1 interleaved pattern of local (sliding window of 1024 tokens) and global attention layers. It also specifies the use of a SigLIP vision encoder with a 'Pan & Scan' algorithm and a MobileNet-v5 based encoder for audio. While the high-level methodology is clear, the exact weights for the 'Mix-and-Match' sub-model extraction are facilitated through a tool (MatFormer Lab) rather than being fully static in the base weights.

Google discloses that the model was trained on approximately 11 trillion tokens across 140+ languages, including web text, code, mathematics, and multimodal data (images/audio). However, specific percentage breakdowns of the dataset composition are not provided. While filtering methodologies for CSAM and sensitive data are mentioned, the exact sources of the 'diverse collection of web text' remain proprietary, and the labeling methods for instruction tuning are undisclosed.

The model uses a SentencePiece tokenizer with a vocabulary size of 262,144 entries, which is consistent with Gemini 2.0. It supports byte-level encodings and split digits, specifically optimized for multilingual balance across 140+ languages. The tokenizer is publicly accessible via Hugging Face and official Google repositories, allowing for direct verification of tokenization behavior and vocabulary alignment.

There is significant transparency regarding the distinction between 'raw' and 'effective' parameters. The E2B variant has a total of approximately 5-6 billion parameters but is designed to operate with an effective footprint of ~2 billion (specifically 1.91B) through PLE and parameter skipping. However, the '6B' total count in the provided metadata slightly conflicts with some technical reports citing 5B for E2B, and the exact active parameter count during multimodal inference vs. text-only inference is described qualitatively rather than with a precise per-task breakdown.

The technical report mentions the use of TPUv5e hardware for training the Gemma 3 family, but specific compute hours, total energy consumption, or carbon footprint data for the E2B variant are not disclosed. While the hardware type is known, the lack of detailed resource metrics or environmental impact calculations results in a low score.

Google provides a comprehensive list of benchmark results (MMLU, GSM8K, HumanEval, etc.) with specified shot counts (e.g., 0-shot, 5-shot). However, the exact evaluation code and specific prompts used for the multimodal benchmarks are not fully public in a single reproducible repository. Third-party verification is available through platforms like LMSYS (LMArena), but the internal 'MatFormer Lab' results for custom slices are harder to verify independently.

The model consistently identifies itself as 'Gemma' and is transparent about its versioning (Gemma 3n) and capabilities. It does not exhibit identity confusion with competitor models like GPT-4. Documentation clearly outlines its role as an edge-optimized multimodal model, and it maintains a coherent identity across various deployment platforms (Ollama, Hugging Face, LiteRT).

The model is released under the 'Google Gemma License,' which allows for both commercial and non-commercial use, subject to specific usage guidelines and redistributable terms. While the license is permissive and clear, it is a custom 'open weights' license rather than a standard OSI-approved open-source license (like Apache 2.0), which imposes some restrictions on redistribution and attribution.

Hardware requirements are well-documented, specifically highlighting the 2GB VRAM target for the E2B variant when using quantization and PLE. Documentation provides specific guidance for deployment on mobile (LiteRT) and consumer GPUs. There is also clear disclosure regarding the memory scaling for its 32,768 token context window and the trade-offs between full-precision (BF16) and quantized (GGUF) execution.

The model uses basic versioning (v1.0), and changelogs are available on Hugging Face. However, as a relatively new release, there is limited historical data on performance drift or a formal long-term deprecation policy. The 'Mix-and-Match' nature of the architecture also introduces potential for 'silent' variation if users deploy different sub-model slices without clear version tracking of the specific slice configuration.