Qwen3-8B

The model is explicitly identified as a dense decoder-only transformer. Architectural details are well-documented in the Qwen3 Technical Report (arXiv:2505.09388), including the use of Grouped Query Attention (GQA), SwiGLU activation, RoPE, and RMSNorm with pre-normalization. A specific refinement, 'qk layernorm', is documented for training stability. The training methodology is detailed as a three-stage process: general pre-training (S1), reasoning optimization (S2), and long-context adaptation (S3).

While the total token count (36 trillion) and the number of languages (119) are clearly stated, the specific breakdown of the dataset (e.g., exact percentages of web, code, and books) is not provided. The documentation mentions general categories like STEM, coding, and synthetic data (distilled from Qwen2.5-Math/Coder), but lacks a detailed public composition breakdown or access to sample data for verification.

The tokenizer is publicly available via Hugging Face and is based on the tiktoken implementation of byte-level Byte Pair Encoding (BBPE). The vocabulary size is precisely stated as 151,669. Documentation confirms its multilingual support for 119 languages and provides clear examples of its application in both 'thinking' and 'non-thinking' modes.

The model's parameter counts are transparently disclosed: 8.2 billion total parameters and 6.95 billion non-embedding parameters. The architecture is clearly defined as dense, distinguishing it from the MoE variants in the same family. Detailed layer and head counts (36 layers, 32 query heads, 8 KV heads) are provided in the technical report.

There is a significant lack of transparency regarding the specific compute resources used. While the hardware types (A100/H100) are implied by the scale of the project and mentioned in community fine-tuning guides, the official documentation does not disclose total GPU hours, energy consumption, or the carbon footprint associated with the 36-trillion-token training run.

The technical report provides scores across standard benchmarks (MMLU, GPQA, GSM8K, etc.) and names the specific versions used. However, while evaluation results are detailed, the full evaluation code and the exact prompts/few-shot examples required for exact reproduction are not centrally hosted in a single, easily accessible repository, though some integration exists in frameworks like OpenCompass.

The model consistently identifies itself as part of the Qwen series. It maintains a clear distinction between its 'thinking' and 'non-thinking' modes, which are documented features rather than identity hallucinations. Versioning is clear (Qwen3-8B), and the model does not attempt to mimic competitors in its official documentation or weights.

The model weights and associated code are released under the Apache 2.0 license, which is a standard, highly permissive open-source license. This allows for both commercial and non-commercial use, derivative works, and redistribution without the restrictive 'custom' terms often found in other 'open' weights releases.

Hardware requirements are well-documented by both the provider and the community. VRAM requirements for FP16 (~16-18GB) and various quantization levels (e.g., Q4_K_M requiring ~5-8GB) are publicly available. Documentation also addresses the memory scaling impact of its 128K context window and the use of YaRN for extension.

The model follows a basic versioning scheme, but there is limited public documentation regarding a formal changelog for weight updates or a structured deprecation policy. While major releases are announced via blog posts and GitHub, the tracking of minor 'silent' updates or performance drift over time lacks a rigorous, transparent framework.