Hunyuan Standard
Active Parameters
52B
Context Length
30K
Modality
Text
Architecture
Mixture of Experts
KV Cache Rate
160 KB/tok
License
Tencent Hunyuan Community License Agreement
Release Date
10 Jun 2024
Knowledge Cutoff
-
API Pricing (per 1M)
Self-hosted only
Hardware Sizing
Precision
Context
Workload Memory: 825.18 GB
FP16 · 8K context
Base Weights: 821.43 GB
KV Cache: 1.25 GB
Runtime & Graphs: 2.50 GB
Minimum Required Hardware
8x NVIDIA H200 141GB
825.2 / 1015.2 GB usable · TP 8
Determine requirements for datacenter hardware:
Upgrade to Pro
Architecture Diagram
Evaluation Benchmarks
Rank
#174
| Benchmark | Score | Rank |
|---|---|---|
General Text | 1311 | 164 |
Rankings
Overall Rank
#174
Coding Rank
-
About Hunyuan Standard
Hunyuan-Large (MoE-A52B) is Tencent's open-source 389B Mixture-of-Experts model activating 52B parameters for advanced bilingual reasoning and text generation. It incorporates Cross-Layer Attention and Grouped-Query Attention across a 256K context window.
Technical Specifications
Attention
Attention Structure
Multi-Head Attention
Attention Heads
80
Key-Value Heads
8
Position Embedding
Absolute Position Embedding
Activation Function
SwigLU
Dimensions
Hidden Dimension Size
6,400
Number of Layers
64
Mixture of Experts
Total Expert Parameters
389.0B
Number of Experts
17
Active Experts
2
About Hunyuan
Tencent Hunyuan large language models with various capabilities.