Technical Specifications

System Requirements

VRAM requirements for different quantization methods and context sizes

Qwen3-30B-A3B

The Qwen3-30B-A3B model, developed by Alibaba, is a Mixture-of-Experts (MoE) language model within the Qwen3 series, designed for efficient inference across a range of natural language processing tasks. It encompasses 30.5 billion parameters in total, with an active subset of approximately 3.3 billion parameters engaged per token during inference. This architectural strategy aims to achieve performance levels comparable to larger dense models while significantly reducing the computational overhead required for each processing step. This model is part of a dual architecture strategy by Qwen 3, which includes both dense and sparse (MoE) designs, providing flexibility for various computational resources and use-case complexities.

Architecturally, Qwen3-30B-A3B is structured with 48 layers and employs a Grouped Query Attention (GQA) mechanism, featuring 32 query heads and 4 key/value heads. The MoE configuration integrates 128 experts, with 8 experts activated per token, and does not incorporate shared experts. A distinctive attribute is its hybrid reasoning system, which enables dynamic transitions between a 'thinking mode' for complex logical reasoning, mathematics, and coding tasks, and a 'non-thinking mode' for general-purpose dialogue. This design allows the model to adapt its computational approach based on task requirements, thereby optimizing resource utilization. The model's foundation rests on a pre-training corpus of 36 trillion tokens, covering 119 languages, which contributes to its extensive multilingual proficiency.

Qwen3-30B-A3B is engineered to process text inputs and is designed to enhance reasoning, instruction-following, and agent capabilities. Its native context window supports up to 32,768 tokens, which can be extended to 131,072 tokens through the application of the YaRN (Yet another RoPE N) method for handling longer sequences. The model utilizes Rotary Position Embedding (RoPE) and incorporates architectural refinements such as global-batch load balancing loss for MoE models and qk layer normalization. These refinements contribute to improved training stability and overall performance. The model is also designed to be fine-tunable for specific downstream applications.