Active Parameters
80B
Context Length
66K
Modality
Reasoning
Architecture
Mixture of Experts
KV Cache Rate
96 KB/tok
License
Apache-2.0
Release Date
1 Feb 2026
Knowledge Cutoff
Jun 2025
API Pricing (per 1M)
Input: $0.09 · Output: $1.10
Precision
Context
Workload Memory: 299.41 GB
FP16 · 8K context
Base Weights: 296.16 GB
KV Cache: 0.75 GB
Runtime & Graphs: 2.50 GB
8x NVIDIA H100 80GB
299.4 / 576.0 GB usable · TP 8
Determine requirements for your specific workload:
Open in VRAM Calculator
Checkpoint format
FP16
CUDA
>= 11.8
PyTorch
>= 2.1
Transformers
>= 4.57.0.dev0
| Architecture | Precision | Attention kernel |
|---|---|---|
Blackwell, Hopper | Native FP16 | FlashAttention-3 |
Ada Lovelace, Ampere | Native FP16 | FlashAttention-2 |
Rank
#141
| Benchmark | Score | Rank |
|---|---|---|
Professional Knowledge | 0.806 | 29 |
Graduate-Level QA | 0.729 | 81 |
General Text | 1399 | 123 |
Coding Index | 0.17 | 135 |
Intelligence Index | auto 0.11 Standard 0.10 | 244 254 |
Overall Rank
#141
Coding Rank
#147
Qwen3-Next-80B-A3B is a sparse Mixture-of-Experts model by Alibaba activating 3B parameters via hybrid Gated DeltaNet linear attention. It delivers ultra-fast sequence modeling and complex mathematical reasoning over an extensible 1M context window.
Attention
Attention Structure
Multi-Head Attention
Attention Heads
16
Key-Value Heads
2
Attention Head Dimension
256
Position Embedding
Absolute Position Embedding
RoPE Theta
10,000,000
Normalization
RMS Normalization
Activation Function
SwigLU
Dimensions
Base Precision
FP16
Hidden Dimension Size
2,048
Number of Layers
48
FFN Intermediate Size (Dense)
512
Tokenizer
Vocabulary Size
151,936
Mixture of Experts
Total Expert Parameters
79.0B
Number of Experts
512
Active Experts
10
FFN Intermediate Size (per Expert)
512
The Alibaba Qwen 3 model family comprises dense and Mixture-of-Experts (MoE) architectures, with parameter counts from 0.6B to 235B. Key innovations include a hybrid reasoning system, offering 'thinking' and 'non-thinking' modes for adaptive processing, and support for extensive context windows, enhancing efficiency and scalability.