Technical Specifications

System Requirements

VRAM requirements for different quantization methods and context sizes

ERNIE-4.5-21B-A3B-Base

The ERNIE-4.5-21B-A3B-Base model is a constituent of Baidu's ERNIE 4.5 family, designed as a powerful, text-focused Mixture-of-Experts (MoE) base model. While the broader ERNIE 4.5 family undergoes joint training on both textual and visual modalities, this specific variant has its text-related parameters extracted, optimizing it for natural language tasks. Its architectural design leverages a heterogeneous MoE structure, which incorporates modality-isolated routing, router orthogonal loss, and multimodal token-balanced loss to ensure effective representation and learning across modalities, even if this variant is primarily for text completion.

This model is engineered for computational efficiency and high performance, supporting a long context length of up to 131,072 tokens. Its MoE architecture distributes the total 21 billion parameters across 64 experts, with 6 active experts per token during generation steps. This design facilitates efficient resource utilization and scalable deployment, benefiting from techniques such as intra-node expert parallelism, memory-efficient pipeline scheduling, and FP8 mixed-precision training. The emphasis on efficiency extends to inference, with strategies like multi-expert parallel collaboration and convolutional code quantization enabling 4-bit/2-bit lossless quantization.

The primary use case for the ERNIE-4.5-21B-A3B-Base model is general-purpose language understanding and generation tasks. It is optimized for Chinese and English text processing, making it suitable for applications requiring robust text completion and comprehension. The model's foundation on the PaddlePaddle deep learning framework further ensures high-performance inference and simplified deployment across various hardware platforms.

No evaluation benchmarks for ERNIE-4.5-21B-A3B-Base available.