Qwen3.5-35B-A3B

The model's architecture is extensively documented in official Hugging Face model cards and technical blog posts. It is a hybrid Gated DeltaNet and sparse Mixture-of-Experts (MoE) transformer. Documentation specifies 40 layers with a 10x block layout (3x Gated DeltaNet -> MoE followed by 1x Gated Attention -> MoE). It details linear attention head counts (32 for V, 16 for QK) and head dimensions (128). The pre-training methodology involves a three-stage process (General, Reasoning, and Long Context) which is publicly described.

Alibaba discloses that the model was trained on approximately 36 trillion tokens across 119 languages. While general categories like web data, PDF-like documents (extracted via Qwen2.5-VL), and synthetic data (generated by Qwen2.5-Math/Coder) are mentioned, there is no granular breakdown of specific data sources or exact percentage compositions. The filtering methodology is described at a high level (multilingual annotation system labeling for educational value and safety), but specific datasets remain proprietary.

The tokenizer is publicly available via the Hugging Face repository and is compatible with standard libraries like Transformers and vLLM. It uses a Byte Pair Encoding (BPE) scheme with a large vocabulary of 151,646 tokens (padded to 248,320). Documentation explicitly states the inclusion of functional control tokens (<|im_start|>, <|im_end|>) and supports 201 languages/dialects, which is verifiable through the provided configuration files.

Transparency regarding parameter density is exemplary for an MoE model. The provider explicitly distinguishes between the 35B total parameters and the 3B active parameters per token. The MoE structure is detailed as having 256 total experts, with 8 routed experts and 1 shared expert activated per forward pass. This level of detail prevents the common 'parameter inflation' marketing trap and provides clear technical specs for compute estimation.

Information regarding the specific compute resources used for training is almost entirely absent. While the training stages and token counts are disclosed, there is no public data on GPU/TPU hours, hardware specifications used for the run, total energy consumption, or carbon footprint. This is a significant gap in an otherwise technical profile.

The model provides results for several standard benchmarks (MMLU-Pro: 85.3%, GPQA Diamond: 84.2%, SWE-bench: 69.2%). While evaluation results are listed on Hugging Face, the specific evaluation code and exact prompt templates used for these official scores are not fully centralized in a single reproducible repository. Third-party verification from platforms like Artificial Analysis and community tests on r/LocalLLaMA provide some external validation, but official reproduction instructions are limited.

The model demonstrates high identity consistency, correctly identifying its version and family in official documentation and API responses. It is transparent about its nature as a mixture-of-experts model and its multimodal capabilities. There are no reported instances of the model claiming to be a competitor's product or misrepresenting its 3B active parameter count as a 35B dense model.

The model is released under the Apache 2.0 license, which is a standard, permissive open-source license. The license file is explicitly included in the Hugging Face repository and allows for commercial use, modification, and distribution without conflicting proprietary terms. This is the highest level of licensing transparency possible.

Hardware requirements are well-documented by both the provider and the community. Official documentation provides guidance for 8-GPU tensor parallel setups for 262k context. Community documentation (e.g., Unsloth, llama.cpp) provides precise VRAM requirements for various quantization levels (e.g., Q4_K_M requiring ~20GB VRAM, Q8_1 requiring ~37GB). The impact of context length on memory scaling is also documented through community benchmarks.

The model uses a naming convention that includes the version (3.5), but a formal semantic versioning changelog is not prominently maintained. While updates are pushed to Hugging Face (e.g., the March 5 GGUF update), these often rely on commit history rather than a structured, public-facing versioning system with deprecation notices. Tracking silent behavior drift over time remains difficult for end-users.