ApX 标志

趋近智

Hunyuan Lite

活跃参数

7B

上下文长度

250K

模态

Text

架构

Mixture of Experts (MoE)

许可证

Tencent Hunyuan Community License

发布日期

30 Oct 2024

知识截止

-

技术规格

专家参数总数

-

专家数量

-

活跃专家

-

注意力结构

Multi-Head Attention

隐藏维度大小

-

层数

-

注意力头

-

键值头

-

激活函数

-

归一化

-

位置嵌入

Absolute Position Embedding

系统要求

不同量化方法和上下文大小的显存要求

Hunyuan Lite

Hunyuan Lite is a compact, text-based language model developed by Tencent, designed for efficiency and broad deployment across various computational environments. This model variant is part of the larger Hunyuan family, strategically optimized for resource-constrained edge devices such as laptops, smartphones, and smart cabin systems, making advanced AI capabilities more accessible. Its fundamental purpose is to provide robust natural language processing, code generation, and mathematical reasoning within a lightweight framework, catering to a range of applications where computational overhead is a critical consideration.

The architectural foundation of Hunyuan Lite incorporates a Mixture of Experts (MoE) structure, a design choice enabling enhanced performance characteristics while maintaining computational efficiency. This configuration was a significant upgrade, implemented on October 30, 2024, alongside an expanded context window. The model supports an ultra-long context length of 256,000 tokens, facilitating the processing and comprehension of extensive textual inputs, such as entire documents or lengthy conversations. A notable design aspect is its fusion-reasoning capability, which allows for distinct "fast-thinking" and "slow-thinking" modes, adapting its processing strategy to the complexity and required depth of reasoning for a given task.

In terms of operational characteristics, Hunyuan Lite is engineered for general language understanding and generation tasks. It exhibits proficient capabilities in processing and responding to queries related to natural language, mathematical problems, and coding challenges. The model is made available with open weights and associated inference code, fostering its integration into diverse development workflows and facilitating specialized fine-tuning for particular industry requirements. It functions as a text-only model, without inherent support for search or multimedia processing directly integrated into its core capabilities.

关于 Hunyuan

Tencent Hunyuan large language models with various capabilities.


其他 Hunyuan 模型

评估基准

排名适用于本地LLM。

没有可用的 Hunyuan Lite 评估基准。

排名

排名

-

编程排名

-

GPU 要求

完整计算器

选择模型权重的量化方法

上下文大小:1024 个令牌

1k
122k
244k

所需显存:

推荐 GPU