Hunyuan TurboS

Tencent provides a high level of architectural detail in the official technical report (arXiv:2505.23076). The model is explicitly described as a hybrid Transformer-Mamba2 Mixture of Experts (MoE) model. It specifies a 128-layer structure with a precise interleaved pattern: 57 Mamba2 layers, 7 Attention layers (using Grouped-Query Attention), and 64 FFN layers. The report details the 'AMF' (Attention-Mamba2-FFN) and 'MF' (Mamba2-FFN) block patterns, providing a level of transparency rarely seen in proprietary models.

The model was pre-trained on a massive corpus of 16 trillion tokens. While the total token count and the use of 3 million instructions for supervised fine-tuning are disclosed, the specific breakdown of the 16T tokens (e.g., percentage of web, code, books) is not provided in detail. The documentation mentions 'high-quality tokens' and 'STEM-specific data' but lacks a granular composition table or public access to data samples, which is common for large-scale industrial models.

The tokenizer is well-documented and consistent with the Hunyuan-Large model. It features a vocabulary of 128K tokens, consisting of 100K tokens from the tiktoken (OpenAI) base and 28K additional tokens specifically optimized for Chinese language support. Technical metrics such as compression rates (3.13 characters per token) are publicly disclosed, and the tokenizer is accessible via the official GitHub repository for the Hunyuan family.

Tencent is transparent about the model's scale, disclosing both total and active parameters. Hunyuan-TurboS has 560 billion total parameters with 56 billion active parameters per token. The MoE structure is detailed as having 32 experts, with a routing strategy that activates 1 shared expert and 2 specialized experts per token. This clear distinction between dense and sparse parameter counts prevents the common 'parameter inflation' marketing trap.

Information regarding training compute is limited to high-level infrastructure descriptions. While Tencent mentions its 'Xingmai' high-performance network and the ability to support clusters of over 100,000 GPUs, it does not disclose the specific GPU hours, hardware type (e.g., H100 vs. H800), or the carbon footprint associated with training Hunyuan-TurboS specifically. This lack of specific resource disclosure is a significant gap.

The model's performance is documented across 23 automated benchmarks with an average score of 77.9%. While the technical report lists scores for MMLU, GSM8K, and HumanEval, and the model is ranked on the LMSYS Chatbot Arena (#8 globally), the exact evaluation code and full prompt sets used for internal benchmarking are not fully public. However, the release of the 'C3-Bench' and 'ArtifactsBench' by the same team provides some reproducible evaluation frameworks for the broader community.

The model maintains a consistent identity as part of the Tencent Hunyuan family. It correctly identifies its version (e.g., Hunyuan-TurboS-20250416) and its specific 'fast-thinking' vs. 'slow-thinking' (Hunyuan-T1) capabilities. There are no documented cases of the model claiming to be a competitor's product or misrepresenting its origin during standard interactions.

The licensing situation is complex and restrictive. While some components are released under the 'Tencent Hunyuan Community License,' it contains significant geographic restrictions (e.g., not applicable in the EU, UK, or South Korea) and requires explicit permission for entities with over 100 million monthly active users. This is not a standard open-source license and creates ambiguity for global commercial use.

Basic hardware requirements are available through Tencent Cloud documentation and community guides. The model's 56B active parameters suggest a high VRAM requirement (estimated ~112GB for FP16), and the use of GQA and Mamba2 layers is explicitly noted as a strategy to reduce KV-cache memory footprint. However, official documentation lacks a comprehensive quantization-to-accuracy tradeoff table for consumer-grade hardware.

Tencent uses date-based versioning (e.g., 20250416) and provides high-level changelogs during major updates (e.g., May 2025 upgrade). However, as a primarily API-driven model, silent updates and behavioral drift are difficult for users to track independently. There is no public, granular version history that allows users to pin specific weights for long-term consistency.