趋近智
注意力结构
Multi-Head Attention
隐藏维度大小
-
层数
-
注意力头
-
键值头
-
激活函数
-
归一化
Layer Normalization
位置嵌入
Absolute Position Embedding
不同量化方法和上下文大小的显存要求
Cohere Command R is a generative language model optimized for enterprise-scale applications, particularly focusing on long-context tasks, retrieval-augmented generation (RAG), and multi-step tool use. It is designed to enable companies to move beyond proof-of-concept AI into production deployments by balancing efficiency with accuracy. The model offers strong capabilities across 10 major languages of global business, including English, French, Spanish, Italian, German, Portuguese, Japanese, Korean, Arabic, and Chinese, with its pre-training data also including many other languages to improve global versatility.
The architecture of Command R is based on an optimized Transformer design, allowing it to handle an extended context window of 128,000 tokens. This long context capability is crucial for processing extensive documents or multi-document conversations, ensuring coherent and contextually grounded responses. The model has been rigorously fine-tuned through supervised fine-tuning (SFT) on instruction-following data and preference tuning, similar to reinforcement learning from human feedback, to align its behavior with user expectations and enhance helpfulness and safety. Command R also features specialized training for grounded generation, allowing it to generate responses with citations from provided document snippets, a key component for robust RAG implementations.
Command R is engineered for practical enterprise use cases, excelling in tasks such as document summarization, question answering, and complex workflow automation. It supports both single-step and multi-step tool use, enabling interaction with external APIs, databases, or search engines. This functionality allows the model to perform actions and integrate with various internal and external systems. Furthermore, the model has demonstrated improved decision-making regarding tool utilization and the ability to follow instructions provided in system messages, along with enhanced structured data analysis.
排名适用于本地LLM。
排名
#49
基准 | 分数 | 排名 |
---|---|---|
Refactoring Aider Refactoring | 0.38 | 12 |
Coding Aider Coding | 0.38 | 15 |
Agentic Coding LiveBench Agentic | 0.02 | 19 |
Coding LiveBench Coding | 0.26 | 27 |
Reasoning LiveBench Reasoning | 0.21 | 28 |
Data Analysis LiveBench Data Analysis | 0.40 | 28 |
Mathematics LiveBench Mathematics | 0.18 | 31 |