Hunyuan T1
Active Parameters
70B (Estimated)
Context Length
32K
Modality
Text
Architecture
Mixture of Experts (MoE)
License
Proprietary
Release Date
22 Aug 2025
Knowledge Cutoff
Dec 2024
API Pricing (per 1M)
Input: $0.14 · Output: $0.56
Evaluation Benchmarks
Rank
#78
| Benchmark | Score | Rank |
|---|---|---|
General Text | 1388 | 106 |
Rankings
Overall Rank
#78
Coding Rank
-
About Hunyuan T1
Hunyuan T1 is Tencent's slow-thinking reasoning engine built on a hybrid Transformer-Mamba MoE architecture for deep logical deduction and STEM problem solving. It utilizes large-scale RL and Cross-Layer Attention over a 256K token context window.
Technical Specifications
Architecture specifications are undisclosed for proprietary models.
Attention
Attention Structure
Multi-Head Attention
Attention Heads
Undisclosed
Key-Value Heads
Undisclosed
Attention Head Dimension
Undisclosed
Position Embedding
Absolute Position Embedding
RoPE Theta
Undisclosed
Sliding Window Attention
Undisclosed
Sliding Window Size
Undisclosed
Sliding Window Ratio
Undisclosed
Linear Attention
Undisclosed
Linear Attention Ratio
Undisclosed
Normalization
RMS Normalization
Activation Function
SwigLU
Dimensions
Hidden Dimension Size
Undisclosed
Number of Layers
128
FFN Intermediate Size (Dense)
Undisclosed
Multi-Token Prediction Heads
Undisclosed
Tokenizer
Vocabulary Size
Undisclosed
Mixture of Experts
Total Expert Parameters
52.0B
Number of Experts
16
Active Experts
2
Shared Experts
Undisclosed
FFN Intermediate Size (per Expert)
Undisclosed
Dense Layers Before MoE
Undisclosed
About Hunyuan
Tencent Hunyuan large language models with various capabilities.