ApX logoApX logo

OLMo 3.1 32B Instruct

Parameters

32B

Context Length

66K

Modality

Text

Architecture

Dense

License

Apache 2.0

Release Date

12 Dec 2025

Knowledge Cutoff

Dec 2024

API Pricing (per 1M)

Input: $0.20 · Output: $0.60

System Requirements

VRAM requirements for different quantization methods and context sizes

1,024 tokens

68.98 GB VRAM

Consumer

4x RTX 4090

24GB VRAM

Datacenter

1x NVIDIA A100

80GB VRAM

Apple Silicon

1x Apple M3 Max

128GB VRAM

65,536 tokens

86.74 GB VRAM

Consumer

4x RTX 4090

24GB VRAM

Datacenter

2x NVIDIA A100

80GB VRAM

Apple Silicon

1x Apple M3 Max

128GB VRAM

Architecture Diagram

Input TokensToken EmbeddingPosition: AbsoluteHidden: 5.1k · Context: 66K · Vocab: 100.3kx 64 layersRMSNormPre-AttentionMulti-Head Attention40Q / 8KV heads · SW: 4.1kHead dim: 128+RMSNormPre-FFNFeed-Forward NetworkSwiGLUIntermediate: 27.6k+Final RMSNormOutput Logits

Evaluation Benchmarks

Rank

#138

BenchmarkScoreRank

General Text

Text Arena

1330

128

Intelligence Index

Artificial Analysis

0.07

207

Rankings

Overall Rank

#138

Coding Rank

-

About OLMo 3.1 32B Instruct

OLMo 3.1 32B Instruct is a fully auditable instruction model developed by Ai2 for conversational AI, agentic tasks, and tool use. It features Grouped-Query Attention and RLVR alignment across a 65K token context window.

Technical Specifications

Attention

Attention Structure

Multi-Head Attention

Attention Heads

40

Key-Value Heads

8

Attention Head Dimension

-

Position Embedding

Absolute Position Embedding

RoPE Theta

500,000

Sliding Window Attention

Yes

Sliding Window Size

4,096

Sliding Window Ratio

-

Linear Attention

-

Linear Attention Ratio

-

Normalization

RMS Normalization

Activation Function

SwigLU

Dimensions

Hidden Dimension Size

5,120

Number of Layers

64

FFN Intermediate Size (Dense)

27,648

Multi-Token Prediction Heads

-

Tokenizer

Vocabulary Size

100,278

About OLMo 3

OLMo (Open Language Model) is a series of fully open language models designed to enable the science of language models. Released by the Allen Institute for AI (Ai2), OLMo 3 provides complete access to training data (Dolma 3), code, checkpoints, logs, and evaluation methodologies. The family includes Base models for pretraining research, Instruct variants for chat and tool use, and Think variants with chain-of-thought reasoning capabilities. All models are trained with staged approach including pretraining, mid-training, and long-context phases.


Other OLMo 3 Models