OLMo 3 7B Instruct

OLMo 3 7B Instruct provides exemplary documentation regarding its architectural lineage. It is explicitly identified as a dense decoder-only Transformer with 7.0B parameters. The training methodology is exhaustively detailed across three distinct stages: general pre-training on Dolma 3, mid-training for capability refinement, and a dedicated context extension stage to reach 65,536 tokens. AI2 provides the exact training scripts (e.g., OLMo-3-1025-7B-pretrain-1.py) and full source code via the OLMo-core GitHub repository, allowing for complete architectural verification.

The model's training data is highly transparent, utilizing the Dolma 3 dataset for pre-training and the Dolci-Instruct datasets for post-training. AI2 discloses the data mix, including web content, scientific papers, and code, and provides public access to the data recipes and curation methodologies. The post-training data (Dolci) is also documented, specifying the use of SFT, DPO, and RLVR stages with associated dataset names. The level of detail regarding filtering and staging is significantly higher than industry standards.

The tokenizer is publicly available through the Hugging Face repository and is integrated into the standard Transformers library. It supports a 65,536-token context window, and the vocabulary size and tokenization approach (Dolma Toolkit) are documented. While specific tokenization alignment studies for all supported languages are less prominent than the architectural details, the public availability of the tokenizer files allows for direct inspection and verification.

The model is clearly stated to be a dense architecture with 7.0 billion parameters. Unlike many competitors, AI2 provides a full architectural breakdown in their technical reports, including the number of layers (32), hidden size (4096), and the number of attention heads (32 Q-heads, 32 KV-heads). There is no ambiguity regarding active vs. total parameters as it is not a Mixture-of-Experts (MoE) model.

AI2 provides rare transparency regarding compute resources, disclosing approximately 234,000 H100 GPU hours for the 7B pre-training phase. They also report energy consumption metrics (~146 MWh for the 7B model) and average power draw (~621W per GPU). This level of detail allows for independent calculation of the carbon footprint and training costs, though the environmental impact data is primarily found in technical reports and community disclosures rather than a consolidated 'green' model card.

Evaluation is highly reproducible through the OLMo-Eval framework and the OLMES evaluation suite. AI2 publishes detailed benchmark results (MMLU-Pro, GPQA, etc.) and provides the code to run these evaluations. While they acknowledge the complexity of few-shot prompting, the release of intermediate checkpoints (step-level) allows researchers to verify performance at various stages of training, which is a significant transparency advantage.

The model consistently identifies itself as part of the OLMo 3 family and is transparent about its nature as an instruction-tuned variant. It does not exhibit the identity confusion common in models that are fine-tuned from competitor bases (like Llama or Qwen), as it is trained from scratch by AI2. The versioning (e.g., 1025-7B) is clearly reflected in the model's metadata and documentation.

The model, weights, and training code are all released under the Apache 2.0 license, which is a standard, permissive open-source license. There are no conflicting commercial restrictions or 'open-weights-only' caveats. AI2's commitment to 'fully open' AI is legally backed by this clear and consistent licensing across all artifacts.

Hardware requirements are well-documented by both AI2 and the community. VRAM requirements for various quantization levels (FP16, Q8, Q4) are available, with specific guidance for consumer hardware (e.g., 24GB for full precision, ~6GB for Q4). The impact of the 65k context window on memory scaling is also noted, though users must rely on community-driven GGUF/EXL2 documentation for the most granular quantization-accuracy tradeoff data.

AI2 maintains a clear versioning system (e.g., OLMo 3 vs 3.1) and provides a changelog in the OLMo-core repository. They are transparent about updates, such as the shift from OLMo 3 to 3.1 to improve reasoning. However, as a research-heavy project, the 'drift' is often presented as progress in new checkpoints rather than a stable API-style deprecation path, which may require users to manually track which specific checkpoint they are using.