Qwen3.5-397B-A17B

The model architecture is extensively documented as a Hybrid Mixture-of-Experts (MoE) with Gated DeltaNet layers. Technical specifications are highly detailed, including the number of layers (60), hidden dimension (4096), and specific hidden layout (15 blocks of 3x Gated DeltaNet to MoE followed by 1x Gated Attention to MoE). The use of 512 total experts with 10 routed and 1 shared expert per token is explicitly stated. While the base model lineage is clear within the Qwen 3.5 family, the specific pre-training methodology is described in high-level technical terms (early fusion multimodal training) rather than a step-by-step procedural paper.

Data transparency is a significant weakness. While the model is described as being trained on 'trillions of multimodal tokens' across 201 languages, specific dataset names, sources, and exact composition percentages (e.g., % code vs % web) are not disclosed. Documentation vaguely refers to 'automated collection' and 'public benchmark datasets' for evaluation, but the pre-training corpus remains a 'black box' with no public sample data or detailed filtering methodology provided.

The tokenizer is publicly available and well-documented with a vocabulary size of 248,320 (often rounded to 250k in documentation). It supports 201 languages and dialects, and its efficiency for non-Latin scripts is highlighted in technical reviews. The vocabulary size and tokenization approach (supporting text, image, and video tokens via early fusion) are verifiable through the official Hugging Face configuration files and community tools like vLLM and SGLang.

Qwen provides exemplary transparency regarding parameter density. It explicitly distinguishes between total parameters (397B) and active parameters (17B). The MoE structure is further detailed with the exact number of experts (512) and the routing mechanism (10 routed + 1 shared). This level of detail prevents the common 'parameter inflation' seen in other MoE models and allows for accurate compute estimation.

There is almost no verifiable information regarding the total compute budget. While the hardware used for inference (H100, H200, B200) is mentioned, the actual training duration, total GPU/TPU hours, and carbon footprint are not disclosed. The documentation mentions 'Next-Generation Training Infrastructure' but lacks the concrete metrics required for a high transparency score.

While the model provides impressive scores on standard benchmarks (MMLU-Pro, GPQA, SWE-bench), the evaluation code is not fully public. Some specific setups are mentioned (e.g., using fixes from Claude 4.5 system cards for TAU2-Bench), but a comprehensive, one-click reproduction repository is missing. Furthermore, there are documented concerns regarding potential contamination in common benchmarks like MATH-500 for the Qwen series, which necessitates a skeptical view of the reported zero-shot gains.

The model demonstrates strong identity consistency, correctly identifying as a Qwen 3.5 series model in most deployments. It is transparent about its 'Thinking Mode' and the use of <think> tags. However, some user reports indicate minor alignment issues where the model may fail to generate answers or exhibit whitespace normalization errors, though it does not typically claim to be a competitor's model.

The model is released under a clear Apache 2.0 license, which is explicitly stated in the GitHub repository, Hugging Face model card, and official blog posts. This license allows for both commercial and non-commercial use, derivative works, and redistribution without the restrictive 'custom' terms often found in other 'open' weights releases.

Hardware requirements are well-documented by both the provider and third-party community members (e.g., Unsloth, NVIDIA). VRAM requirements for various quantization levels (FP16, FP8, 4-bit GGUF) are available, with clear guidance that the full model requires ~807GB on disk. The distinction between total memory for loading (397B) and active compute for inference (17B) is clearly explained for local deployment.

Versioning is handled through standard Hugging Face repository updates, but a formal, detailed changelog or semantic versioning system for the weights themselves is not prominently maintained. While the release date (Feb 2026) is clear, there is limited information on how future 'silent' updates or safety alignment shifts will be communicated to the community.