趋近智
SEA-LION-7B
参数
7.1B
上下文长度
2.048K
模态
Text
架构
Dense
许可证
Apache-2.0
发布日期
1 Dec 2023
训练数据截止日期
-
技术规格
注意力结构
Multi-Head Attention
隐藏维度大小
-
层数
-
注意力头
-
键值头
-
激活函数
-
归一化
-
位置嵌入
Absolute Position Embedding
系统要求
不同量化方法和上下文大小的显存要求
SEA-LION-7B
SEA-LION-7B is a 7.1 billion parameter base model designed for Southeast Asian languages. It is trained on a 1 trillion token corpus including regional news and web content. The model uses the MPT architecture and is released under the Apache 2.0 license.
关于 SEA-LION
Southeast Asian Languages In One Network (SEA-LION) is a family of language models developed by AI Singapore for Southeast Asian languages. The models support English, Indonesian, Malay, Thai, Vietnamese, Tagalog, Burmese, Khmer, Lao, Tamil, and Chinese. It focuses on regional linguistic patterns and is available in base and instruction-tuned variants.
其他 SEA-LION 模型
评估基准
没有可用的 SEA-LION-7B 评估基准。