AREX-2
Parameters
27B
Context Length
262K
Modality
Text
Architecture
Dense
KV Cache Rate
256 KB/tok
License
Apache 2.0
Release Date
29 Sept 2026
Knowledge Cutoff
-
API Pricing (per 1M)
Self-hosted only
Hardware & System Requirements
Precision
Context
Workload Memory: 17.07 GB
INT4 · 8K context
Base Weights: 12.57 GB
KV Cache: 2.00 GB
Runtime & Graphs: 2.50 GB
Minimum Required Hardware
1x NVIDIA L4 24GB
17.1 / 21.6 GB usable
2x RTX 4090 24GB
17.1 / 43.2 GB usable · TP 2
Apple M4 24GB
17.1 / 18.0 GB usable
Determine requirements for your specific workload:
Open in VRAM Calculator
Runtime requirements
Checkpoint format
FP16
CUDA
>= 11.8
PyTorch
>= 2.1
Transformers
>= 5.2.0
| Architecture | Precision | Attention kernel |
|---|---|---|
Blackwell, Hopper | Native FP16 | FlashAttention-3 |
Ada Lovelace, Ampere | Native FP16 | FlashAttention-2 |
Architecture Diagram
Evaluation Benchmarks
No evaluation benchmarks for AREX-2 available.
Rankings
Overall Rank
-
Coding Rank
-
About AREX-2
AREX-2 is an open foundation language model released by BAAI built on enhanced transformer architecture. It is designed for multi-turn conversational interaction, code comprehension, and general language tasks.
Specifications
Attention
Attention Structure
Grouped-Query Attention
Attention Heads
24
Key-Value Heads
4
Attention Head Dimension
256
Position Embedding
ROPE
RoPE Theta
10,000,000
Linear Attention
Yes
Linear Attention Ratio
75.0%
Normalization
RMS Normalization
Activation Function
SwigLU
Dimensions
Base Precision
FP16
Hidden Dimension Size
5,120
Number of Layers
64
FFN Intermediate Size (Dense)
17,408
Multi-Token Prediction Heads
1
Tokenizer
Vocabulary Size
248,320
Resources
About AREX
The AREX model family developed by BAAI.
Other AREX Models
- No related models available