Qwen3.5-122B-A10B

The model's architecture is extensively documented in official Hugging Face and NVIDIA model cards. It utilizes a hybrid 'Gated DeltaNet' (linear attention) and sparse Mixture-of-Experts (MoE) transformer architecture with 48 layers. Specific structural details are provided, including a 3:1 ratio of DeltaNet to standard attention cycles, hidden dimensions of 3072, and a 256-expert MoE setup. While the high-level methodology is clear, a full peer-reviewed technical paper detailing the specific pre-training curriculum or architectural ablation is not yet publicly linked, though a 'coming soon' documentation placeholder exists on GitHub.

Information regarding the training data is extremely limited. Official documentation on NVIDIA and Hugging Face lists the training dataset, collection methodology, and labeling as 'Undisclosed'. While the model claims support for 201 languages and multimodal inputs (text, image, video), there is no public breakdown of the data proportions (e.g., web vs. code) or specific sources used for the early-fusion multimodal training.

The tokenizer is publicly accessible via the Hugging Face repository and integrated into standard libraries like Transformers and vLLM. It uses Byte Pair Encoding (BPE) with a stated vocabulary size of 248,320 tokens (padded). Documentation confirms support for 201 languages and provides specific token-to-character ratios for English and Chinese. Verification by third-party users in local deployment (llama.cpp) confirms the tokenizer's functional integrity.

The model provides exemplary transparency regarding its parameter density. It explicitly states a total of 122B parameters with 10B active parameters per token. The MoE structure is detailed as having 256 experts per layer, with 8 routed experts and 1 shared expert activated per token. This prevents the common 'parameter inflation' marketing trap by clearly distinguishing between total and active weights.

There is virtually no public information regarding the compute resources used to train the model. No GPU/TPU hours, hardware cluster specifications, training duration, or carbon footprint data have been disclosed by Alibaba Cloud. The only hardware mentions relate to inference requirements, not training provenance.

The model provides scores for several standard benchmarks (MMLU-Pro, GPQA Diamond, SWE-bench) and some newer ones (Terminal-Bench 2.0). While evaluation results are detailed in the model card, the specific evaluation code and exact prompts used for these internal results are not fully public. However, the model is available for third-party testing on platforms like Artificial Analysis and OpenRouter, allowing for independent verification of performance claims.

While the model generally identifies as part of the Qwen family in standard instruction tasks, there are documented instances of identity confusion in its internal reasoning traces. Users have reported the model claiming to be 'Gemini' or an API-based service even when running locally. This suggests significant identity drift likely stemming from the training or distillation process, which undermines its self-identification reliability.

The model is clearly released under the Apache 2.0 license, which is a standard, permissive open-source license. This is explicitly stated across all primary distribution channels (Hugging Face, GitHub, Kaggle, and NVIDIA). The terms for commercial and non-commercial use are well-defined by the license itself, providing high legal clarity for developers.

Hardware requirements are well-documented by both the provider and the community. Official specs recommend 8 GPUs for full context (262k) in BF16, while community documentation provides detailed VRAM breakdowns for various quantization levels (Q4_K_M, NVFP4). For example, a Q4_K_M quant is verified to run on ~73GB-80GB of VRAM. The scaling of memory with context length is also noted, with native support up to 262k tokens.

The model uses a clear naming convention (Qwen3.5-122B-A10B), but there is a lack of a formal, public changelog or version history for weight updates. While the release date is clear, there is no established mechanism for tracking silent updates or performance drift over time. The 'coming soon' status of official documentation further limits the transparency of its versioning lifecycle.