Spark X2.5 4B
Parameters
4B
Context Length
1.05M
Modality
Text
Architecture
Dense
KV Cache Rate
144 KB/tok
License
Apache 2.0
Release Date
24 Aug 2026
Knowledge Cutoff
-
API Pricing (per 1M)
Self-hosted only
Hardware Sizing
Precision
Context
Workload Memory: 5.49 GB
INT4 · 8K context
Base Weights: 1.86 GB
KV Cache: 1.13 GB
Runtime & Graphs: 2.50 GB
Minimum Required Hardware
1x NVIDIA L4 24GB
5.5 / 21.6 GB usable
1x RTX 3060 12GB
5.5 / 10.8 GB usable
Apple M4 24GB
5.5 / 18.0 GB usable
Determine requirements for your specific workload:
Open in VRAM Calculator
Architecture Diagram
Evaluation Benchmarks
No evaluation benchmarks for Spark X2.5 4B available.
Rankings
Overall Rank
-
Coding Rank
-
About Spark X2.5 4B
Spark X2.5 4B is a compact, high-efficiency language model designed for edge deployment and fast local inference. It delivers strong multilingual and reasoning performance relative to its lightweight parameter scale.
Technical Specifications
Attention
Attention Structure
Grouped-Query Attention
Attention Heads
16
Key-Value Heads
4
Attention Head Dimension
256
Position Embedding
ROPE
RoPE Theta
5,000,000
Sliding Window Attention
Yes
Sliding Window Size
512
Sliding Window Ratio
75.0%
Normalization
RMS Normalization
Activation Function
GELU
Dimensions
Hidden Dimension Size
2,560
Number of Layers
36
FFN Intermediate Size (Dense)
10,240
Tokenizer
Vocabulary Size
131,072
Resources
About Spark X
The Spark X model series developed by Thinking Machine Labs, featuring lightweight and high-efficiency models optimized for edge deployment, fast local inference, and low memory footprints.