趋近智
活跃参数
7B
上下文长度
250K
模态
Text
架构
Mixture of Experts (MoE)
许可证
Tencent Hunyuan Community License
发布日期
30 Oct 2024
知识截止
-
专家参数总数
-
专家数量
-
活跃专家
-
注意力结构
Multi-Head Attention
隐藏维度大小
-
层数
-
注意力头
-
键值头
-
激活函数
-
归一化
-
位置嵌入
Absolute Position Embedding
不同量化方法和上下文大小的显存要求
Hunyuan Lite is a compact, text-based language model developed by Tencent, designed for efficiency and broad deployment across various computational environments. This model variant is part of the larger Hunyuan family, strategically optimized for resource-constrained edge devices such as laptops, smartphones, and smart cabin systems, making advanced AI capabilities more accessible. Its fundamental purpose is to provide robust natural language processing, code generation, and mathematical reasoning within a lightweight framework, catering to a range of applications where computational overhead is a critical consideration.
The architectural foundation of Hunyuan Lite incorporates a Mixture of Experts (MoE) structure, a design choice enabling enhanced performance characteristics while maintaining computational efficiency. This configuration was a significant upgrade, implemented on October 30, 2024, alongside an expanded context window. The model supports an ultra-long context length of 256,000 tokens, facilitating the processing and comprehension of extensive textual inputs, such as entire documents or lengthy conversations. A notable design aspect is its fusion-reasoning capability, which allows for distinct "fast-thinking" and "slow-thinking" modes, adapting its processing strategy to the complexity and required depth of reasoning for a given task.
In terms of operational characteristics, Hunyuan Lite is engineered for general language understanding and generation tasks. It exhibits proficient capabilities in processing and responding to queries related to natural language, mathematical problems, and coding challenges. The model is made available with open weights and associated inference code, fostering its integration into diverse development workflows and facilitating specialized fine-tuning for particular industry requirements. It functions as a text-only model, without inherent support for search or multimedia processing directly integrated into its core capabilities.
Tencent Hunyuan large language models with various capabilities.
排名适用于本地LLM。
没有可用的 Hunyuan Lite 评估基准。