ERNIE-4.5-300B-A47B-Base

The model's Mixture-of-Experts (MoE) architecture is well-documented in the ERNIE 4.5 Technical Report, detailing a 'multimodal heterogeneous' structure. It specifies the use of modality-isolated routing, router orthogonal loss, and multimodal token-balanced loss to manage expert specialization. The base model is explicitly identified as an extraction of text-related parameters from a larger multimodal pre-training phase, and the use of Grouped Query Attention (GQA) with specific head counts (64 query, 8 KV) is verified in technical documentation.

While Baidu claims the model was trained on 'trillions of tokens' across textual and visual modalities, there is no public breakdown of the dataset composition (e.g., specific percentages of web, code, or academic data). The documentation mentions a 'human-model-in-the-loop' cleaning process but lacks transparency regarding the specific sources or the ratio of synthetic to organic data, falling into the category of vague marketing assertions.

The tokenizer is publicly accessible via the PaddlePaddle and Hugging Face repositories. It uses a SentencePiece-based approach with a confirmed vocabulary size of 103,424 tokens. Documentation explicitly covers special tokens (e.g., mask, bos, eos) and supports both Chinese and English, aligning with the model's stated multilingual capabilities.

Baidu provides exemplary transparency regarding parameter density, clearly distinguishing between the 300 billion total parameters and the 47 billion active parameters per token. The architectural breakdown includes specific details on the FFN expert categorization (text, vision, and shared experts) and the scaling-efficient infrastructure used to manage this density.

Transparency regarding compute is poor. While the technical report mentions a Model FLOPs Utilization (MFU) of 47% and the use of FP8 mixed-precision training on the PaddlePaddle framework, it fails to disclose the total GPU/TPU hours, the specific hardware cluster size, the training duration, or the environmental carbon footprint.

Evaluation code is provided within the ERNIEKit repository, and scores are reported for standard benchmarks like IFEval and MMLU-Pro. However, the score is limited because the primary results rely on internal evaluations with significant skepticism from the research community regarding the lack of independent third-party verification and the absence of direct comparisons with contemporary industry leaders like Gemini 2.0 or GPT-4.5.

The model consistently identifies itself as part of the ERNIE 4.5 family across API responses and documentation. There are no documented instances of the model claiming a competitor's identity or misrepresenting its versioning, and it maintains clear distinction between its 'Base' and 'Thinking' (VL) variants.

The model and its associated development toolkits (ERNIEKit, FastDeploy) are released under the Apache License 2.0. This is a highly transparent, permissive open-source license that explicitly allows for commercial use and derivative works without conflicting proprietary terms.

Baidu provides specific hardware requirements for various deployment scenarios, including VRAM needs for 4-bit (4x 80GB GPUs) and 2-bit (1x 141GB GPU) quantization. The documentation also details the memory scaling for its 131,072-token context window and provides guidance on using the FastDeploy toolkit for optimization.

The model uses semantic versioning (4.5) and maintains a changelog on GitHub. However, because the model was released in mid-2025, there is insufficient public data to track long-term performance drift or the impact of silent 'alignment taxes' over time. The documentation for previous version deprecation is currently minimal.