趋近智
参数
14B
上下文长度
32.768K
模态
Text
架构
Dense
许可证
MIT
发布日期
-
知识截止
Mar 2025
注意力结构
Multi-Head Attention
隐藏维度大小
-
层数
-
注意力头
-
键值头
-
激活函数
-
归一化
-
位置嵌入
Absolute Position Embedding
不同量化方法和上下文大小的显存要求
Microsoft's Phi-4 Reasoning Plus is a 14-billion parameter language model specifically optimized for complex reasoning tasks. It is an enhanced variant within the Phi-4 family, building upon the base Phi-4 model. The model's primary purpose is to address scenarios requiring long-chain reasoning, such as advanced mathematics, scientific inquiry, and code generation. Its development emphasizes delivering high-quality outputs even in computationally constrained environments.
The architecture of Phi-4 Reasoning Plus is a dense, decoder-only Transformer. It incorporates an extended context window of 32,000 tokens to facilitate processing of lengthy reasoning chains. The model utilizes rotary position embeddings to maintain coherence and track token positions effectively across extended sequences. Training involved supervised fine-tuning on a curated dataset of chain-of-thought traces, along with reinforcement learning to further enhance performance. This methodology focuses on high-quality synthetic and filtered organic data, ensuring proficiency in complex problem-solving.
Phi-4 Reasoning Plus demonstrates an increased latency compared to its counterpart, Phi-4 Reasoning, due to its generation of approximately 50% more tokens for more detailed responses. This characteristic makes it particularly suitable for high-accuracy tasks where comprehensive reasoning is paramount. The model is designed to operate efficiently on consumer-grade hardware, including mobile devices, tablets, and desktops, thereby expanding its accessibility for various AI applications.
The Microsoft Phi-4 model family comprises small language models prioritizing efficient, high-capability reasoning. Its development emphasizes robust data quality and sophisticated synthetic data integration. This approach enables enhanced performance and on-device deployment capabilities.
排名适用于本地LLM。
排名
#17
基准 | 分数 | 排名 |
---|---|---|
Graduate-Level QA GPQA | 0.69 | 5 |
Professional Knowledge MMLU Pro | 0.76 | 6 |
Coding LiveBench Coding | 0.61 | 9 |
General Knowledge MMLU | 0.69 | 10 |
Reasoning LiveBench Reasoning | 0.58 | 12 |
Mathematics LiveBench Mathematics | 0.63 | 13 |
Agentic Coding LiveBench Agentic | 0.05 | 14 |
Data Analysis LiveBench Data Analysis | 0.55 | 15 |