Qwen3-1.7B

The model is part of the Qwen3 series, with its architecture extensively documented in the Qwen3 Technical Report (arXiv:2505.09388). It is a dense causal transformer with 28 layers, a hidden dimension of 2048, and 16/8 GQA heads. Specific technical refinements such as SwiGLU, RMSNorm with pre-normalization, and QK-Norm for long-context stability are explicitly detailed. The training methodology is described as a three-stage process (General, Reasoning, and Long Context), though specific hyperparameters for each stage are not fully disclosed.

Alibaba discloses that the model was trained on approximately 36 trillion tokens covering 119 languages. The documentation mentions broad categories (web data, books, PDFs) and the use of synthetic data generated by Qwen2.5-Math and Qwen2.5-Coder. While the three-stage curriculum is explained (e.g., Stage 2 increasing STEM/coding proportions), the exact percentage breakdown of the 36T tokens across these categories is not provided, and the specific data sources remain proprietary.

The tokenizer is publicly available via the Qwen GitHub and Hugging Face repositories. It uses Byte-Level Byte-Pair Encoding (BBPE) with a vocabulary size of 151,669 (or 151,936 including special tokens). It supports 119 languages and includes specific tokens for its 'thinking' mode (<think>...</think>). The alignment between the tokenizer and the claimed multilingual support is verifiable through the provided technical report and model files.

The model is explicitly defined as a dense architecture with 1.70 billion total parameters. Documentation further clarifies that it contains 1.4 billion non-embedding parameters. This level of precision, distinguishing between total and non-embedding counts, is exemplary for transparency in parameter density.

While the technical report mentions that the training was 'large-scale' and utilized 'significant resources,' it fails to provide specific GPU/TPU hours, hardware cluster specifications, or the total energy consumption/carbon footprint. There are no public estimates for the training cost or duration, which is a significant gap in transparency regarding the model's environmental and resource impact.

The technical report provides extensive benchmark results across standard sets like MMLU, GSM8K, and HumanEval, often comparing performance in 'thinking' vs 'non-thinking' modes. However, while some evaluation settings (like temperature and top-p) are recommended in the model card, the full evaluation code and exact few-shot prompts used to generate the official report scores are not fully public, limiting independent verification.

The model consistently identifies as part of the Qwen series and is transparent about its dual-mode (thinking/non-thinking) capabilities. It provides clear versioning within the Qwen3 family. There are no documented instances of the model claiming to be a competitor's product or misrepresenting its 1.7B scale.

The model and its weights are released under the Apache 2.0 license, which is a standard, permissive open-source license. This is explicitly stated in the official blog, the technical report, and the Hugging Face repository, with no conflicting proprietary terms found in the documentation.

Hardware requirements are well-documented, with specific VRAM estimates provided for different quantization levels (e.g., ~4.74 GB for FP16/Base). The impact of context length on memory (KV cache scaling) is addressed, and third-party verification on consumer hardware (RTX 3060/3090) is widely available and consistent with official claims.

The model follows the Qwen team's standard release cycle, but there is no formal public changelog or semantic versioning for minor weight updates. While major versions are clear (Qwen2.5 to Qwen3), tracking subtle behavior drift or silent updates to the 'thinking' mode logic is difficult for end-users.