ApX logoApX logo

Command R Plus

Parameters

104B

Context Length

128K

Modality

Text

Architecture

Dense

License

CC-BY-NC

Release Date

4 Apr 2024

Knowledge Cutoff

Feb 2023

Technical Specifications

Attention Structure

Multi-Head Attention

Hidden Dimension Size

12288

Number of Layers

64

Attention Heads

96

Key-Value Heads

8

Activation Function

-

Normalization

Layer Normalization

Position Embedding

Absolute Position Embedding

Command R Plus

Command R Plus is a large-scale generative model engineered by Cohere to support the most demanding enterprise-grade artificial intelligence applications. It is specifically architected to handle complex, multi-step agentic workflows and sophisticated Retrieval Augmented Generation (RAG) at scale. The model is designed to bridge the gap between experimental prototypes and production-ready systems by offering a high degree of reliability, particularly in grounding responses through inline citations and managing extensive conversation histories. Its multilingual capabilities are extensive, featuring robust performance across ten primary global business languages and training exposure to over twenty-three languages, ensuring its utility in diverse international operational environments.

From a technical perspective, Command R Plus utilizes a dense, decoder-only transformer architecture with 104 billion parameters. It incorporates Grouped Query Attention (GQA) to optimize inference latency and memory efficiency, which is essential for processing its expansive 128,000-token context window. The model employs Rotary Positional Embeddings (RoPE) and Layer Normalization to ensure stable training and precise token dependency management over long sequences. The alignment process involves a rigorous combination of Supervised Fine-Tuning (SFT) and preference training, which refines the model's ability to follow complex system instructions and utilize external tools with a high success rate, including the capability for self-correction during tool failure.

Operational performance is a hallmark of the Command R Plus variant, which has seen significant updates to improve throughput and reduce latency without increasing hardware requirements. It is optimized for high-volume production workloads where cost-efficiency and speed are as critical as accuracy. The model's design facilitates seamless integration into existing business ecosystems, such as CRM and ERP systems, where it can automate structured data analysis and execute multi-step reasoning tasks. By providing open weights under a non-commercial license, Cohere enables researchers and developers to deploy and inspect the model's capabilities in private environments, fostering a transparent approach to enterprise AI deployment.

About Command


Other Command Models

Evaluation Benchmarks

Rank

#66

BenchmarkScoreRank

Web Development

WebDev Arena

1262

48

Rankings

Overall Rank

#66

Coding Rank

#67

GPU Requirements

Full Calculator

Choose the quantization method for model weights

Context Size: 1,024 tokens

1k
63k
125k

VRAM Required:

Recommended GPUs

Command R Plus: Specifications and GPU VRAM Requirements