Qwen 3.8 27B

Parameters

27B

Context Length

262K

Modality

Multimodal

Auxiliary Parameters

450M

Architecture

Dense

KV Cache Rate

256 KB/tok

License

Apache 2.0

Release Date

5 Aug 2026

Knowledge Cutoff

-

API Pricing (per 1M)

Input: $0.50 · Output: $3.00

Hardware & System Requirements

Precision

Native

Context

Workload Memory: 17.07 GB

INT4 · 8K context

Base Weights: 12.57 GB

KV Cache: 2.00 GB

Runtime & Graphs: 2.50 GB

Minimum Required Hardware

1x NVIDIA L4 24GB

17.1 / 21.6 GB usable

2x RTX 4090 24GB

17.1 / 43.2 GB usable · TP 2

Apple M4 24GB

17.1 / 18.0 GB usable

Determine requirements for your specific workload:

Open in VRAM Calculator

Runtime requirements

Checkpoint format

FP16

CUDA

>= 11.8

PyTorch

>= 2.1

Transformers

>= 5.8.0.dev0

ArchitecturePrecisionAttention kernel

Blackwell, Hopper

Native FP16

FlashAttention-3

Ada Lovelace, Ampere

Native FP16

FlashAttention-2

Architecture Diagram

Input TokensToken EmbeddingPosition: RoPEHidden: 5.1k · Context: 262K · Vocab: 248.3kx 64 layersRMSNormPre-AttentionGrouped-Query Attention24Q / 4KV headsHead dim: 256+RMSNormPre-FFNFeed-Forward NetworkSwiGLUIntermediate: 17.4k+Final RMSNormOutput Logits

Evaluation Benchmarks

Rank

#50

BenchmarkScoreRank

Agentic Coding

LiveBench Agentic

0.61

11

Agentic Index

Artificial Analysis
max

0.46

medium

0.44

low

0.39

Standard

0.22

27

28

44

77

Graduate-Level QA

GPQA

0.892

27

0.77

28

0.75

33

Web Development

WebDev Arena

1590

33

LiveBench Average

LiveBench Average

0.75

34

0.76

48

0.86

48

0.80

50

max

0.68

medium

0.56

low

0.58

Standard

0.45

52

75

70

98

Agent Arena

Agent Arena

-4.76

57

General Text

Text Arena

1434

92

Intelligence Index

Artificial Analysis
max

0.20

medium

0.17

low

0.16

Standard

0.10

171

197

210

249

Rankings

Overall Rank

#50

Coding Rank

#43

About Qwen 3.8 27B

Qwen 3.8 27B is a 27-billion parameter dense foundation model by Alibaba that offers balanced efficiency and strong reasoning performance. It excels at multi-turn conversational tasks, programming, and long-context understanding.

Specifications

Attention

Attention Structure

Grouped-Query Attention

Attention Heads

24

Key-Value Heads

4

Attention Head Dimension

256

Position Embedding

ROPE

RoPE Theta

10,000,000

Linear Attention

Yes

Linear Attention Ratio

75.0%

Normalization

RMS Normalization

Activation Function

SwigLU

Dimensions

Base Precision

FP16

Auxiliary Parameters

450M

Hidden Dimension Size

5,120

Number of Layers

64

FFN Intermediate Size (Dense)

17,408

Multi-Token Prediction Heads

1

Tokenizer

Vocabulary Size

248,320

About Qwen 3.8

Alibaba's Qwen 3.8 generation represents the frontier hybrid Mixture-of-Experts architecture designed for coding, professional work, research, and long-horizon agentic tasks. It features a 2.4-trillion parameter architecture (95B active per token) combining Gated DeltaNet linear attention with standard Gated Attention, available both as open weights and as a hosted flagship service.


Other Qwen 3.8 Models