OLMo 3.1 32B Think

Parameters

32B

Context Length

66K

Modality

Text

Architecture

Dense

KV Cache Rate

256 KB/tok

License

Apache 2.0

Release Date

12 Dec 2025

Knowledge Cutoff

Dec 2024

API Pricing (per 1M)

Input: $0.15 · Output: $0.50

Hardware & System Requirements

Precision

Native

Context

Workload Memory: 19.40 GB

INT4 · 8K context

Base Weights: 14.90 GB

KV Cache: 2.00 GB

Runtime & Graphs: 2.50 GB

Minimum Required Hardware

1x NVIDIA L4 24GB

19.4 / 21.6 GB usable

2x RTX 4090 24GB

19.4 / 43.2 GB usable · TP 2

Apple M3 Pro 36GB

19.4 / 27.0 GB usable

Determine requirements for your specific workload:

Open in VRAM Calculator

Runtime requirements

Checkpoint format

FP16

CUDA

>= 11.8

PyTorch

>= 2.1

Transformers

>= 4.57.1

ArchitecturePrecisionAttention kernel

Blackwell, Hopper

Native FP16

FlashAttention-3

Ada Lovelace, Ampere

Native FP16

FlashAttention-2

Architecture Diagram

Input TokensToken EmbeddingPosition: AbsoluteHidden: 5.1k · Context: 66K · Vocab: 100.3kx 64 layersRMSNormPre-AttentionMulti-Head Attention40Q / 8KV heads · SW: 4.1kHead dim: 128+RMSNormPre-FFNFeed-Forward NetworkSwiGLUIntermediate: 27.6k+Final RMSNormOutput Logits

Evaluation Benchmarks

Rank

#200

BenchmarkScoreRank

General Text

Text Arena
auto

1286

173

Intelligence Index

Artificial Analysis
auto

0.07

293

Rankings

Overall Rank

#200

Coding Rank

-

About OLMo 3.1 32B Think

OLMo 3.1 32B Think is a large-scale open reasoning model from Ai2 optimized for complex mathematics and code logic via verifiable rewards. It generates inspectable reasoning chains over a 65K context window under Apache 2.0.

Specifications

Attention

Attention Structure

Multi-Head Attention

Attention Heads

40

Key-Value Heads

8

Attention Head Dimension

128

Position Embedding

Absolute Position Embedding

RoPE Theta

500,000

Sliding Window Attention

Yes

Sliding Window Size

4,096

Normalization

RMS Normalization

Activation Function

SwigLU

Dimensions

Base Precision

FP16

Hidden Dimension Size

5,120

Number of Layers

64

FFN Intermediate Size (Dense)

27,648

Tokenizer

Vocabulary Size

100,278

About OLMo 3

OLMo (Open Language Model) is a series of fully open language models designed to enable the science of language models. Released by the Allen Institute for AI (Ai2), OLMo 3 provides complete access to training data (Dolma 3), code, checkpoints, logs, and evaluation methodologies. The family includes Base models for pretraining research, Instruct variants for chat and tool use, and Think variants with chain-of-thought reasoning capabilities. All models are trained with staged approach including pretraining, mid-training, and long-context phases.


Other OLMo 3 Models