Technical Specifications

System Requirements

VRAM requirements for different quantization methods and context sizes

Qwen3-30B-A3B

The Qwen3-30B-A3B model, developed by Alibaba, is a Mixture-of-Experts (MoE) language model within the Qwen3 series. Its architecture is optimized for efficient inference across a range of natural language processing tasks. The model totals 30.5 billion parameters, with an active set of approximately 3.3 billion parameters engaged per token during inference, a design choice aimed at achieving performance comparable to larger dense models while significantly reducing computational overhead.

Architecturally, Qwen3-30B-A3B is structured with 48 layers and employs a Grouped Query Attention (GQA) mechanism, featuring 32 query heads and 4 key/value heads. The MoE configuration includes 128 experts, with 8 experts activated per token, and does not incorporate shared experts. A notable feature is its hybrid reasoning system, allowing for seamless transitions between a 'thinking mode' for complex logical reasoning, mathematics, and coding tasks, and a 'non-thinking mode' for general-purpose dialogue. This design enables the model to adapt its computational strategy to the demands of the task, ensuring optimal resource utilization. The model is built upon a pre-training corpus of 36 trillion tokens, encompassing 119 languages, thereby expanding its multilingual proficiency.

Qwen3-30B-A3B is engineered to process text inputs and is designed to enhance reasoning, instruction-following, and agent capabilities. Its native context window supports up to 32,768 tokens, which can be extended to 131,072 tokens through the application of the YaRN (Yet another RoPE N) method for handling longer sequences. The model leverages Rotary Position Embedding (RoPE) and integrates refinements such as global-batch load balancing loss for MoE models and qk layer normalization, which contribute to improved training stability and performance. It is designed to be fine-tunable for specific use cases.