ApX logoApX logo

LFM2.5 24B A2B

Active Parameters

24B

Context Length

131K

Modality

Text

Architecture

Mixture of Experts

KV Cache Rate

80 KB/tok

License

other

Release Date

12 Oct 2026

Knowledge Cutoff

-

API Pricing (per 1M)

Self-hosted only

Hardware & System Requirements

Precision

Native

Context

Workload Memory: 15.47 GB

INT4 · 8K context

Base Weights: 12.34 GB

KV Cache: 0.63 GB

Runtime & Graphs: 2.50 GB

Minimum Required Hardware

1x NVIDIA L4 24GB

15.5 / 21.6 GB usable

2x RTX 4090 24GB

15.5 / 43.2 GB usable · TP 2

Apple M4 24GB

15.5 / 18.0 GB usable

Determine requirements for your specific workload:

Open in VRAM Calculator

Runtime requirements

Checkpoint format

FP16

CUDA

>= 11.8

PyTorch

>= 2.1

Transformers

>= 5.12.1

ArchitecturePrecisionAttention kernel

Blackwell, Hopper

Native FP16

FlashAttention-3

Ada Lovelace, Ampere

Native FP16

FlashAttention-2

Architecture Diagram

Evaluation Benchmarks

No evaluation benchmarks for LFM2.5 24B A2B available.

Rankings

Overall Rank

-

Coding Rank

-

About LFM2.5 24B A2B

LFM2.5 24B A2B is a mixture-of-experts model from Liquid AI activating 2 billion parameters per token. It leverages Liquid's non-transformer dynamical architecture designed for high efficiency and extended sequence modeling.

Specifications

Attention

Attention Structure

Grouped-Query Attention

Attention Heads

32

Key-Value Heads

8

Attention Head Dimension

64

Position Embedding

ROPE

RoPE Theta

10,000,000

Linear Attention

Yes

Linear Attention Ratio

75.0%

Normalization

RMS Normalization

Activation Function

SwigLU

Dimensions

Base Precision

FP16

Hidden Dimension Size

2,048

Number of Layers

40

FFN Intermediate Size (Dense)

11,776

Tokenizer

Vocabulary Size

128,000

Mixture of Experts

Total Expert Parameters

2.5B

Number of Experts

64

Active Experts

4

FFN Intermediate Size (per Expert)

1,536

Dense Layers Before MoE

2

About LFM2

The LFM2 model family developed by Liquid AI.


Other LFM2 Models