GPT-OSS 120B

Open Source

Open Weights

Active Parameters

117B

Context Length

128K

Modality

Text

Architecture

Mixture of Experts (MoE)

License

Apache 2.0

Release Date

5 Aug 2025

Knowledge Cutoff

Jun 2024

API Pricing (per 1M)

Input: $0.15 · Output: $0.59

System Requirements

VRAM requirements for different quantization methods and context sizes

1,024 tokens

247.28 GB VRAM

Consumer

13x RTX 4090

24GB VRAM

Datacenter

4x NVIDIA A100

80GB VRAM

Apple Silicon

3x Apple M3 Max

128GB VRAM

128,000 tokens

257.11 GB VRAM

Consumer

13x RTX 4090

24GB VRAM

Datacenter

4x NVIDIA A100

80GB VRAM

Apple Silicon

3x Apple M3 Max

128GB VRAM

Architecture Diagram

Evaluation Benchmarks

Rank

#118

Benchmark	Score	Rank
Professional Knowledge MMLU Pro	high 0.807	27
Software Engineering SWE-bench Verified	0.26	36
Graduate-Level QA GPQA	high 0.809 Standard 0.801	64 66
Agentic Index Artificial Analysis	high 0.06	107
General Text Text Arena	1352	118
Coding Index Artificial Analysis	high 0.30 low 0.21	122 131
Intelligence Index Artificial Analysis	high 0.12 low 0.10	176 187
Summarization Archived ProLLM Summarization	0.976	🥈 2
General Knowledge Reference MMLU	0.9	🥉 3
Coding Archived Aider Coding	high 0.42	17

Rankings

Overall Rank

#118

Coding Rank

#109

About GPT-OSS 120B

GPT-OSS 120B is a large open-weight model from OpenAI, designed to operate in data centers and on high-end desktops and laptops. It is developed to support advanced reasoning, agentic tasks, and diverse developer use cases, functioning as a text-only model for both input and output modalities.

Technical Specifications

Attention

Attention Structure

Multi-Head Attention

Attention Heads

Key-Value Heads

Attention Head Dimension

Position Embedding

Absolute Position Embedding

RoPE Theta

150,000

Sliding Window Attention

Yes

Sliding Window Size

128

Sliding Window Ratio

Linear Attention

Linear Attention Ratio

Normalization

RMS Normalization

Activation Function

SwigLU

Dimensions

Hidden Dimension Size

2,880

Number of Layers

FFN Intermediate Size (Dense)

2,880

Multi-Token Prediction Heads

Tokenizer

Vocabulary Size

201,088

Mixture of Experts

Total Expert Parameters

5.1B

Number of Experts

128

Active Experts

Shared Experts

FFN Intermediate Size (per Expert)

Dense Layers Before MoE

Resources

Official Documentation Release Notes Download Weights Source Code

About GPT-OSS

Open-weight language models from OpenAI.

Other GPT-OSS Models

GPT-OSS 20B