OLMo 3.1 32B Think

The model's architecture is extensively documented in the official technical report and Hugging Face model card. It is a dense decoder-only Transformer with 64 layers, a hidden dimension of 5120, and 40 query heads with 8 KV heads (GQA). It utilizes SwiGLU activations, RMSNorm, and RoPE with YaRN-style scaling for a 65,536 token context window. The training methodology is fully described as a multi-stage process (pretraining, mid-training, long-context, SFT, DPO, and RLVR), and the training code is publicly available in the OLMo-core and open-instruct repositories.

OLMo 3.1 32B Think is trained on the Dolma 3 dataset, which is publicly released and documented. The pretraining mix (5.9T tokens from a 9.3T crawl) and the specialized Dolci-Think-RL datasets for post-training are disclosed. Documentation includes details on data filtering, quality-aware upsampling (top 5% upsampled 7x), and de-contamination procedures against benchmark test sets. The full 'model flow' including data points is part of the release.

The tokenizer is publicly available via the Hugging Face repository and is compatible with standard libraries like Transformers and vLLM. While the exact vocabulary size is not explicitly highlighted in all marketing summaries, it is verifiable through the provided configuration files and code. The tokenizer is aligned with the training data and supports the claimed 65k context window.

The model is explicitly stated to be a dense 32B parameter model, avoiding the ambiguity often found in Mixture-of-Experts (MoE) models. The architectural breakdown (layers, hidden size, GQA configuration) is clearly provided, allowing for precise calculation of active parameters and memory requirements.

Ai2 provides significant detail regarding compute resources. The 32B model pretraining required approximately 1.05 million H100 GPU hours, with an additional 21 days on 224 GPUs for the 3.1 RLVR stage. Power consumption was measured at ~649W per GPU during pretraining, totaling ~681MWh for the 32B model. This level of environmental and resource disclosure is exemplary.

Evaluation is conducted using the open-source OLMo-Eval framework. Results are provided for standard benchmarks (AIME, ZebraLogic, IFEval, GSM8K, etc.) with specific versioning. While the technical report provides high-level methodology, the full evaluation code and exact prompt configurations are accessible through the linked GitHub repositories, though third-party verification is still maturing for the 3.1 variant.

The model consistently identifies as part of the OLMo 3.1 family and is transparent about its 'Think' variant nature, which uses explicit <think> tags for chain-of-thought reasoning. It does not attempt to mimic competitor identities and maintains clear versioning (e.g., 3.1 vs 3.0) in its system prompts and documentation.

The model, weights, and training code are all released under the Apache 2.0 license, which is a standard, permissive open-source license. There are no conflicting proprietary terms or 'open-weights-only' restrictions that limit commercial use or derivative works, making it one of the most legally transparent models available.

Hardware requirements are well-documented, with specific VRAM estimates for different precisions (FP16 requires ~64GB, while 4-bit/8-bit quantizations are noted for 24GB-32GB cards). Context length memory scaling for the 65k window is also addressed, providing clear guidance for deployment on consumer vs. enterprise hardware.

The model uses clear semantic versioning (3.1) to distinguish it from the initial 3.0 release. Ai2 maintains a public 'model flow' that tracks checkpoints and updates. While a formal changelog in the style of software is less centralized, the transition from 3.0 to 3.1 is well-documented in blog posts and technical updates, including the specific additional RL training days.