Gemma 3 1B

Gemma 3 1B is extensively documented in the official technical report (arXiv:2503.19786), which details its decoder-only transformer architecture. It specifies a 5:1 interleaving of local (sliding window) and global attention layers to manage KV-cache efficiency, a 32K context window, and the use of Grouped-Query Attention (GQA) with QK-norm. The training methodology, including distillation from larger teacher models and post-training via RLHF, RLMF, and RLEF, is clearly described.

While Google discloses that the model was trained on 2 trillion tokens and provides general categories (web documents, code, mathematics), it lacks a precise percentage breakdown of the dataset composition. The documentation mentions support for 140+ languages and describes filtering processes for CSAM and sensitive data, but the specific data sources and their proportions remain proprietary, falling short of high-transparency standards.

The model uses the same SentencePiece tokenizer as Gemini 2.0, which is publicly available and well-documented. It features a vocabulary size of 262,144 entries, optimized for multilingual support across 140+ languages. Technical details such as byte-level encodings, split digits, and preserved whitespace are explicitly stated in the technical report and verifiable via the Hugging Face model repository.

The model is explicitly identified as a dense 1.0B parameter model. Technical documentation provides a clear architectural breakdown, including the number of layers (26) and hidden dimensions (1152). Unlike MoE models, there is no ambiguity regarding active vs. total parameters, though detailed weight distribution across specific components (e.g., FFN vs. Attention) requires manual calculation from the provided config files.

Google discloses the hardware used (TPUv5e) and the scale of the cluster (512 chips), but it does not provide the total GPU/TPU hours, energy consumption, or a calculated carbon footprint for the training run. While the infrastructure type is known, the lack of specific resource duration or environmental impact data limits the score to the lower end of moderate transparency.

The technical report provides scores for standard benchmarks (MMLU, GSM8K, HumanEval, etc.) and specifies some evaluation settings (e.g., 0-shot vs. few-shot). However, the full evaluation code and exact prompts used for all benchmarks are not publicly released in a centralized repository, making exact third-party reproduction difficult without significant reverse-engineering of the described methodology.

Gemma 3 1B demonstrates strong identity consistency, correctly identifying its version and capabilities in official documentation and model cards. It is transparent about its text-only nature compared to the multimodal larger variants in the same family. There are no documented cases of the model misrepresenting itself as a competitor's product or denying its AI nature.

The model is released under the 'Gemma Terms of Use,' which is a custom permissive license rather than a standard OSI-approved open-source license like Apache 2.0. While it allows for commercial use and redistribution, it includes specific 'Prohibited Use' policies and 'viral' clauses regarding model derivatives that create legal complexity for developers, distinguishing it from true open-source software.

Hardware requirements are exceptionally well-documented. Google provides VRAM estimates for various precision levels (BF16: 1.5GB, SFP8: 1.1GB, Q4: 892MB) and explicitly mentions that the model can run on devices with as little as 4GB of RAM. The impact of Quantization-Aware Training (QAT) on maintaining accuracy while reducing footprint is also detailed in official blog posts and technical guides.

Google maintains a 'Gemma releases' page that tracks version history and release dates (e.g., March 12, 2025). However, it lacks a detailed, granular changelog for minor weight updates or specific documentation on performance drift over time. While major versions are clear, the transparency regarding incremental behavioral changes is moderate.