Qwen3.5-2B
Parameters
2B
Context Length
262K
Modality
Multimodal
Auxiliary Parameters
350M
Architecture
Dense
KV Cache Rate
48 KB/tok
License
Apache 2.0
Release Date
24 Feb 2026
Knowledge Cutoff
-
API Pricing (per 1M)
Self-hosted only
Hardware Sizing
Precision
Context
Workload Memory: 3.81 GB
INT4 · 8K context
Base Weights: 0.93 GB
KV Cache: 0.38 GB
Runtime & Graphs: 2.50 GB
Minimum Required Hardware
1x NVIDIA L4 24GB
3.8 / 21.6 GB usable
1x RTX 3060 12GB
3.8 / 10.8 GB usable
Apple M4 24GB
3.8 / 18.0 GB usable
Determine requirements for your specific workload:
Open in VRAM Calculator
Architecture Diagram
Evaluation Benchmarks
Rank
#192
| Benchmark | Score | Rank |
|---|---|---|
Professional Knowledge | 0.665 | 49 |
Graduate-Level QA | 0.516 | 101 |
Coding Index | 0.02 | 157 |
Intelligence Index | auto 0.07 Standard 0.06 | 274 294 |
Rankings
Overall Rank
#192
Coding Rank
#165
About Qwen3.5-2B
Qwen3.5-2B is Alibaba Cloud's small-scale multimodal foundation model with 2B parameters, released February 2026. It uses a hybrid architecture combining Gated Delta Networks and Gated Attention in a 6×(3×DeltaNet→FFN→1×Attention→FFN) pattern. In thinking mode, it achieves MMLU-Pro (74.0%), GPQA Diamond (65.8%), and GPQA (51.6%). Features unified vision-language capabilities, 262k native context, multi-token prediction training, and supports both thinking and non-thinking modes for prototyping, fine-tuning, and research purposes across 201 languages.
Technical Specifications
Attention
Attention Structure
Grouped-Query Attention
Attention Heads
8
Key-Value Heads
2
Attention Head Dimension
256
Position Embedding
ROPE
RoPE Theta
10,000,000
Linear Attention
Yes
Linear Attention Ratio
75.0%
Normalization
RMS Normalization
Activation Function
SwigLU
Dimensions
Auxiliary Parameters
350M
Hidden Dimension Size
2,048
Number of Layers
24
FFN Intermediate Size (Dense)
6,144
Multi-Token Prediction Heads
1
Tokenizer
Vocabulary Size
248,320
About Qwen 3.5
Qwen 3.5 is Alibaba Cloud's latest-generation foundation model family, released February 2026. It represents a significant leap forward, integrating breakthroughs in multimodal learning (unified vision-language foundation), efficient hybrid architecture (Gated Delta Networks with sparse Mixture-of-Experts), scalable reinforcement learning across million-agent environments, and global linguistic coverage spanning 201 languages. Available under Apache 2.0 license with open weights.