趋近智
DeepSeek-V3.1
活跃参数
671B
上下文长度
128K
模态
Text
架构
Mixture of Experts (MoE)
许可证
MIT License
发布日期
21 Aug 2025
训练数据截止日期
-
技术规格
专家参数总数
37.0B
专家数量
257
活跃专家
8
注意力结构
Multi-Head Attention
隐藏维度大小
7168
层数
61
注意力头
-
键值头
-
激活函数
SwigLU
归一化
RMS Normalization
位置嵌入
ROPE
系统要求
不同量化方法和上下文大小的显存要求
DeepSeek-V3.1
A hybrid model that supports both "thinking" and "non-thinking" modes for chat, reasoning, and coding. It's a Mixture-of-Experts (MoE) model with a massive context length and efficient architecture.
关于 DeepSeek-V3
DeepSeek-V3 is a Mixture-of-Experts (MoE) language model comprising 671B parameters with 37B activated per token. Its architecture incorporates Multi-head Latent Attention and DeepSeekMoE for efficient inference and training. Innovations include an auxiliary-loss-free load balancing strategy and a multi-token prediction objective, trained on 14.8T tokens.
其他 DeepSeek-V3 模型
评估基准
排名适用于本地LLM。
排名
#5
| 基准 | 分数 | 排名 |
|---|---|---|
Data Analysis LiveBench Data Analysis | 0.73 | 🥈 2 |
Graduate-Level QA GPQA | 0.80 | 🥈 2 |
Coding Aider Coding | 0.72 | 5 |
Reasoning LiveBench Reasoning | 0.62 | 6 |
StackUnseen ProLLM Stack Unseen | 0.48 | 6 |
Web Development WebDev Arena | 1359.84 | 6 |
Mathematics LiveBench Mathematics | 0.82 | 7 |
Professional Knowledge MMLU Pro | 0.85 | 8 |
General Knowledge MMLU | 0.68 | 11 |