Qwen3.5-35B-A3B

Active Parameters

35B

Context Length

262K

Modality

Multimodal

Auxiliary Parameters

450M

Architecture

Mixture of Experts

KV Cache Rate

80 KB/tok

License

Apache 2.0

Release Date

24 Feb 2026

Knowledge Cutoff

-

API Pricing (per 1M)

Input: $0.14 · Output: $1.00

Hardware & System Requirements

Precision

Native

Context

Workload Memory: 20.82 GB

INT4 · 8K context

Base Weights: 17.70 GB

KV Cache: 0.63 GB

Runtime & Graphs: 2.50 GB

Minimum Required Hardware

1x NVIDIA L4 24GB

20.8 / 21.6 GB usable

2x RTX 4090 24GB

20.8 / 43.2 GB usable · TP 2

Apple M3 Pro 36GB

20.8 / 27.0 GB usable

Determine requirements for your specific workload:

Open in VRAM Calculator

Runtime requirements

Checkpoint format

FP16

CUDA

>= 11.8

PyTorch

>= 2.1

Transformers

>= 4.57.0.dev0

ArchitecturePrecisionAttention kernel

Blackwell, Hopper

Native FP16

FlashAttention-3

Ada Lovelace, Ampere

Native FP16

FlashAttention-2

Architecture Diagram

Input TokensToken EmbeddingPosition: RoPEHidden: 2k · Context: 262K · Vocab: 248.3kx 40 layersRMSNormPre-AttentionGrouped-Query Attention16Q / 2KV headsHead dim: 256+RMSNormPre-FFNSparse MoE FFN (9/256 experts)SwiGLUIntermediate: 512+Final RMSNormOutput Logits

Evaluation Benchmarks

Rank

#129

BenchmarkScoreRank

Professional Knowledge

MMLU Pro

0.853

10

Graduate-Level QA

GPQA

0.842

48

0.37

112

Web Development

WebDev Arena

1253

117

General Text

Text Arena

1394

126

Intelligence Index

Artificial Analysis
auto

0.19

Standard

0.15

186

222

Rankings

Overall Rank

#129

Coding Rank

#131

About Qwen3.5-35B-A3B

Qwen3.5-35B-A3B is Alibaba Cloud's efficient multimodal foundation model, released February 2026. With 35B total parameters and 3B activated through a Mixture-of-Experts architecture (256 experts), it delivers strong performance with minimal compute. It achieves MMLU-Pro (85.3%), GPQA Diamond (84.2%), SWE-bench Verified (69.2%), and Terminal-Bench 2.0 (40.5%). Qwen3.5-Flash is the hosted API version. Features unified vision-language capabilities, 262k native context (extensible to 1M), and strong performance on multimodal reasoning, coding, and multilingual tasks.

Specifications

Attention

Attention Structure

Grouped-Query Attention

Attention Heads

16

Key-Value Heads

2

Attention Head Dimension

256

Position Embedding

ROPE

RoPE Theta

10,000,000

Linear Attention

Yes

Linear Attention Ratio

75.0%

Normalization

RMS Normalization

Activation Function

SwigLU

Dimensions

Base Precision

FP16

Auxiliary Parameters

450M

Hidden Dimension Size

2,048

Number of Layers

40

FFN Intermediate Size (Dense)

512

Multi-Token Prediction Heads

1

Tokenizer

Vocabulary Size

248,320

Mixture of Experts

Total Expert Parameters

3.0B

Number of Experts

256

Active Experts

9

FFN Intermediate Size (per Expert)

512

About Qwen 3.5

Qwen 3.5 is Alibaba Cloud's latest-generation foundation model family, released February 2026. It represents a significant leap forward, integrating breakthroughs in multimodal learning (unified vision-language foundation), efficient hybrid architecture (Gated Delta Networks with sparse Mixture-of-Experts), scalable reinforcement learning across million-agent environments, and global linguistic coverage spanning 201 languages. Available under Apache 2.0 license with open weights.


Other Qwen 3.5 Models