趋近智
OLMo 3 7B Think
参数
7B
上下文长度
65.536K
模态
Text
架构
Dense
许可证
Apache 2.0
发布日期
25 Oct 2025
训练数据截止日期
Dec 2024
技术规格
注意力结构
Multi-Head Attention
隐藏维度大小
4096
层数
32
注意力头
32
键值头
32
激活函数
SwigLU
归一化
-
位置嵌入
Absolute Position Embedding
系统要求
不同量化方法和上下文大小的显存要求
OLMo 3 7B Think
The OLMo 3 7B Think model is a specialized variant within the OLMo 3 family, developed by the Allen Institute for AI (Ai2). This model is engineered to address complex problems requiring multi-step logical inference by making its reasoning process transparent. It is designed to surface intermediate thinking steps, providing researchers and developers with explicit thinking tokens to examine the model's internal deliberations before reaching a final answer. This capability supports enhanced interpretability and auditability of AI systems.
Architecturally, OLMo 3 7B Think is a Transformer-style autoregressive language model with a dense architecture, comprising 7 billion parameters. It utilizes a multi-headed attention mechanism and incorporates Rotary Position Embeddings (RoPE) with scaling to support an extended context length of up to 65,536 tokens. The model's training methodology involves a multi-stage approach. It is initially pre-trained on the comprehensive Dolma 3 dataset and subsequently post-trained through Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), and Reinforcement Learning from Verifiable Rewards (RLVR) on custom Dolci-Think datasets. This layered training focuses on imbuing the model with robust reasoning skills, particularly in domains such as mathematics and coding, while ensuring the model's 'thought process' is explicitly generated.
This variant is optimized for reasoning-intensive tasks, providing a capable foundation for academic research and practical Natural Language Processing (NLP) workflows that demand transparent problem-solving. Its design allows for efficient, inspectable reasoning capabilities, making advanced AI accessible on more modest hardware. The full transparency of the OLMo project, which includes the release of all training data, code, checkpoints, and associated training details under an Apache 2.0 license, fosters reproducibility and further scientific inquiry into model development and behavior.
关于 OLMo 3
OLMo (Open Language Model) is a series of fully open language models designed to enable the science of language models. Released by the Allen Institute for AI (Ai2), OLMo 3 provides complete access to training data (Dolma 3), code, checkpoints, logs, and evaluation methodologies. The family includes Base models for pretraining research, Instruct variants for chat and tool use, and Think variants with chain-of-thought reasoning capabilities. All models are trained with staged approach including pretraining, mid-training, and long-context phases.
其他 OLMo 3 模型
评估基准
排名适用于本地LLM。
没有可用的 OLMo 3 7B Think 评估基准。