Hunyuan TurboS
Active Parameters
560B (Estimated)
Context Length
32K
Modality
Text
Architecture
Mixture of Experts (MoE)
License
Proprietary
Release Date
16 Jul 2025
Knowledge Cutoff
Dec 2024
API Pricing (per 1M)
Input: $0.11 · Output: $0.28
Evaluation Benchmarks
Rank
#90
| Benchmark | Score | Rank |
|---|---|---|
General Text | 1383 | 106 |
Rankings
Overall Rank
#90
Coding Rank
-
About Hunyuan TurboS
Hunyuan TurboS is a high-performance enterprise model by Tencent utilizing a hybrid Transformer-Mamba2 MoE framework with adaptive Chain-of-Thought paths. Pretrained on 16T tokens, it delivers fast inference and deep reasoning over a 256K context.
Technical Specifications
Architecture specifications are undisclosed for proprietary models.
Attention
Attention Structure
Multi-Head Attention
Attention Heads
64
Key-Value Heads
8
Attention Head Dimension
Undisclosed
Position Embedding
Absolute Position Embedding
RoPE Theta
Undisclosed
Sliding Window Attention
Undisclosed
Sliding Window Size
Undisclosed
Sliding Window Ratio
Undisclosed
Linear Attention
Undisclosed
Linear Attention Ratio
Undisclosed
Normalization
RMS Normalization
Activation Function
SwigLU
Dimensions
Hidden Dimension Size
5,120
Number of Layers
128
FFN Intermediate Size (Dense)
Undisclosed
Multi-Token Prediction Heads
Undisclosed
Tokenizer
Vocabulary Size
Undisclosed
Mixture of Experts
Total Expert Parameters
56.0B
Number of Experts
32
Active Experts
3
Shared Experts
Undisclosed
FFN Intermediate Size (per Expert)
Undisclosed
Dense Layers Before MoE
Undisclosed
Resources
About Hunyuan
Tencent Hunyuan large language models with various capabilities.