Gemma 3 4B is explicitly documented as a decoder-only transformer model with a hybrid attention mechanism. The technical report details a 5:1 interleaving ratio of local sliding window self-attention (1,024 token window) and global self-attention layers. It specifies the use of a 400M parameter SigLIP vision encoder and a 'Pan&Scan' algorithm for handling varying image aspect ratios. The training methodology, including the use of knowledge distillation from larger Gemini models, is publicly disclosed in the official technical report (arXiv:2503.19786).

Google provides high-level categories for the training data, including web documents (140+ languages), code, mathematics, and images. The 4B model was trained on 4 trillion tokens. However, specific dataset sources, proportions of each category, and detailed filtering/cleaning methodologies beyond general safety filtering (CSAM and sensitive data) are not disclosed. The reliance on 'proprietary' mixtures and distillation from closed models limits full transparency into the data provenance.

The model uses the Gemini 2.0 tokenizer, which is a SentencePiece-based tokenizer with a vocabulary size of 262,208 tokens. It is publicly available via Hugging Face and integrated into the 'transformers' library. Documentation confirms support for over 140 languages and provides details on how the tokenizer handles multimodal inputs by reserving specific token slots for image embeddings.

The model is a dense architecture with 4.0 billion parameters. Detailed architectural specifications are available, including 30 layers, a hidden dimension of 2048, 32 attention heads, and 8 key-value heads (Grouped-Query Attention). While it is a dense model, the documentation clearly distinguishes it from the sparse or 'Matryoshka' variants (Gemma 3n) released in the same family, preventing parameter inflation confusion.

Information regarding training compute is minimal. While the technical report mentions the use of TPUv5e hardware for training, it does not disclose the total TPU/GPU hours, energy consumption, or carbon footprint. Cost estimates and environmental impact data are conspicuously absent from official documentation, falling into the 'low' transparency category for this pillar.

Google provides comprehensive benchmark results across standard suites (MMLU, GSM8K, HumanEval, etc.) and multimodal benchmarks (DocVQA, MMMU). However, while the technical report describes the evaluation settings (e.g., 0-shot vs few-shot), the exact prompts and full evaluation code are not consistently provided in a single reproducible repository, making third-party verification dependent on independent implementations like 'lm-evaluation-harness'.

Gemma 3 4B consistently identifies itself as a Google-developed model and maintains clear versioning between its 'PT' (Pre-trained) and 'IT' (Instruction-tuned) variants. It does not exhibit identity confusion with competitor models and is transparent about its multimodal limitations (e.g., the 1B variant being text-only while the 4B+ variants are multimodal).

The model is released under the 'Gemma Terms of Use,' which is a custom 'open weights' license rather than a standard OSI-approved license like Apache 2.0. It permits commercial use but includes specific restrictions, such as prohibiting the use of the model to train other models (distillation) and requiring attribution. The terms are clear but more restrictive than true open-source licenses.

Hardware requirements are exceptionally well-documented. Official guides provide VRAM estimates for various quantization levels (FP16, SFP8, Q4_0) and context lengths. For example, the 4B model is noted to require ~9.2 GB VRAM for text tasks in BF16 and ~3.4 GB in 4-bit quantization. The impact of the 128K context window on KV-cache memory is also detailed, showing a reduction in overhead due to the hybrid attention architecture.

While the model uses clear naming conventions (Gemma 3 4B IT/PT), there is no public, detailed changelog for minor weight updates or 'silent' refreshes. Users have reported configuration issues in early releases (e.g., missing top-level config fields in Hugging Face) that required manual patching until library updates were pushed, indicating some friction in version tracking and deployment stability.