Active Parameters
24B
Context Length
131K
Modality
Text
Architecture
Mixture of Experts
KV Cache Rate
80 KB/tok
License
other
Release Date
12 Oct 2026
Knowledge Cutoff
-
API Pricing (per 1M)
Self-hosted only
Precision
Context
Workload Memory: 15.47 GB
INT4 · 8K context
Base Weights: 12.34 GB
KV Cache: 0.63 GB
Runtime & Graphs: 2.50 GB
1x NVIDIA L4 24GB
15.5 / 21.6 GB usable
2x RTX 4090 24GB
15.5 / 43.2 GB usable · TP 2
Apple M4 24GB
15.5 / 18.0 GB usable
Determine requirements for your specific workload:
Open in VRAM Calculator
Checkpoint format
FP16
CUDA
>= 11.8
PyTorch
>= 2.1
Transformers
>= 5.12.1
| Architecture | Precision | Attention kernel |
|---|---|---|
Blackwell, Hopper | Native FP16 | FlashAttention-3 |
Ada Lovelace, Ampere | Native FP16 | FlashAttention-2 |
No evaluation benchmarks for LFM2.5 24B A2B available.
Overall Rank
-
Coding Rank
-
LFM2.5 24B A2B is a mixture-of-experts model from Liquid AI activating 2 billion parameters per token. It leverages Liquid's non-transformer dynamical architecture designed for high efficiency and extended sequence modeling.
Attention
Attention Structure
Grouped-Query Attention
Attention Heads
32
Key-Value Heads
8
Attention Head Dimension
64
Position Embedding
ROPE
RoPE Theta
10,000,000
Linear Attention
Yes
Linear Attention Ratio
75.0%
Normalization
RMS Normalization
Activation Function
SwigLU
Dimensions
Base Precision
FP16
Hidden Dimension Size
2,048
Number of Layers
40
FFN Intermediate Size (Dense)
11,776
Tokenizer
Vocabulary Size
128,000
Mixture of Experts
Total Expert Parameters
2.5B
Number of Experts
64
Active Experts
4
FFN Intermediate Size (per Expert)
1,536
Dense Layers Before MoE
2
The LFM2 model family developed by Liquid AI.