ERNIE 5.5 Flash Base
Active Parameters
280B
Context Length
1.05M
Modality
Text
Architecture
Mixture of Experts
KV Cache Rate
86 KB/tok
License
other
Release Date
21 Oct 2026
Knowledge Cutoff
-
API Pricing (per 1M)
Self-hosted only
Hardware Sizing
Precision
Context
Workload Memory: 550.79 GB
FP16 · 8K context
Base Weights: 547.62 GB
KV Cache: 0.67 GB
Runtime & Graphs: 2.50 GB
Minimum Required Hardware
8x NVIDIA H100 80GB
550.8 / 576.0 GB usable · TP 8
Determine requirements for your specific workload:
Open in VRAM Calculator
Architecture Diagram
Evaluation Benchmarks
No evaluation benchmarks for ERNIE 5.5 Flash Base available.
Rankings
Overall Rank
-
Coding Rank
-
About ERNIE 5.5 Flash Base
ERNIE 5.5 Flash Base is an efficient Mixture-of-Experts foundation language model developed by Baidu. It is engineered for high throughput and rapid inference across general text comprehension and instruction workflows.
Technical Specifications
Attention
Attention Structure
Compressed Sparse Attention 2
Attention Heads
64
Key-Value Heads
1
Attention Head Dimension
512
Position Embedding
ROPE
RoPE Theta
8,000,000
Sliding Window Attention
Yes
Sliding Window Size
128
KV Latent Dimension
576
Indexer Key Dimension
128
Normalization
RMS Normalization
Activation Function
SwigLU
Dimensions
Hidden Dimension Size
4,096
Number of Layers
43
FFN Intermediate Size (Dense)
14,336
Multi-Token Prediction Heads
1
Tokenizer
Vocabulary Size
201,216
Mixture of Experts
Total Expert Parameters
14.0B
Number of Experts
512
Active Experts
10
Shared Experts
1
FFN Intermediate Size (per Expert)
2,048
Dense Layers Before MoE
1
Resources
About ERNIE 5.5
The ERNIE 5.5 model family developed by Baidu.
Other ERNIE 5.5 Models
- No related models available