Hunyuan A13B

The model's architecture is extensively documented in an official technical report and GitHub repository. It is a decoder-only Transformer utilizing a sparse Mixture-of-Experts (MoE) design with 64 non-shared experts and 1 shared expert. Key architectural modifications like Grouped Query Attention (GQA), SwiGLU activation, and a dual-mode 'fast/slow' reasoning framework are clearly described. The pretraining procedure, including a three-stage process (foundation, fast annealing, and long-context adaptation), is publicly detailed.

Tencent discloses that the model was trained on a 20 trillion token corpus with a specific 250 billion token STEM-focused subset. While the report mentions general categories such as math textbooks, GitHub code, and scientific texts, it lacks a precise percentage-based breakdown of the entire 20T corpus. The data cleaning and filtering methodology (e.g., 'refined knowledge labeling system') is mentioned but lacks the granular detail required for a higher score.

The tokenizer is publicly available via the official Hugging Face repository and supports a vocabulary size of 128,000 tokens. It is consistent with previous Hunyuan models and is documented to support multilingual capabilities. The implementation is verifiable through the provided `tokenizer_config.json` and integration with standard libraries like `transformers` and `vLLM`.

The model provides exemplary transparency regarding its MoE parameters. It explicitly states a total of 80 billion parameters with 13 billion active parameters per token (1 shared expert + 8 routed experts). The architectural breakdown (32 layers, 64 experts) is clearly defined in the technical report and configuration files, leaving no ambiguity about dense vs. sparse counts.

While the technical report describes the training stages and scaling laws used, it conspicuously lacks specific details on the hardware hours (GPU/TPU hours), the exact cluster specifications used for the 20T token training, and the associated carbon footprint or environmental impact data. This information is largely withheld for proprietary or competitive reasons.

Tencent provides results for numerous standard benchmarks (MMLU, MATH, GSM8K) and has released two new evaluation datasets (ArtifactsBench and C3-Bench) to the community. However, while the technical report exists, the exact evaluation scripts and full prompt templates for all reported scores are not fully centralized in a way that ensures 1:1 third-party reproduction without significant effort.

The model demonstrates high identity consistency, correctly identifying itself as Hunyuan-A13B and maintaining version awareness. There are no documented instances of the model claiming to be a competitor's product (e.g., GPT-4). It is transparent about its MoE nature and its specific 'thinking' modes during interaction.

The licensing situation is complex and potentially misleading. While marketing materials and some repository files mention 'Apache 2.0', the primary weights are governed by the 'Tencent Hunyuan Community License Agreement'. This custom license includes significant restrictions, such as territorial limitations (excluding the EU, UK, and South Korea) and prohibitions on using the model to improve other AI models, which contradicts standard open-source definitions.

Hardware requirements are well-documented for various deployment scenarios. The repository provides VRAM estimates for FP16, FP8, and INT4 quantization. It also includes specific guidance on memory scaling for the 256K context window and suggests configurations for consumer-grade hardware (e.g., RTX 4090) versus datacenter GPUs.

The model uses basic versioning, and a changelog is present in the GitHub repository. However, the history is relatively short, and there is limited information on long-term drift or a formal deprecation policy for older weight checkpoints. Updates appear to be released as new variants rather than a continuous semantic versioning stream.