Step 3.5 Flash
Active Parameters
196.81B
Context Length
256K
Modality
Multimodal
Architecture
Mixture of Experts
KV Cache Rate
180 KB/tok
License
Apache 2.0
Release Date
11 Feb 2026
Knowledge Cutoff
-
API Pricing (per 1M)
Input: $0.10 · Output: $0.30
Hardware & System Requirements
Precision
Context
Workload Memory: 390.98 GB
FP16 · 8K context
Base Weights: 387.08 GB
KV Cache: 1.41 GB
Runtime & Graphs: 2.50 GB
Minimum Required Hardware
8x NVIDIA H100 80GB
391.0 / 576.0 GB usable · TP 8
Determine requirements for your specific workload:
Open in VRAM Calculator
Runtime requirements
Checkpoint format
FP16
CUDA
>= 11.8
PyTorch
>= 2.1
Transformers
>= 4.40.1
| Architecture | Precision | Attention kernel |
|---|---|---|
Blackwell, Hopper | Native FP16 | FlashAttention-3 |
Ada Lovelace, Ampere | Native FP16 | FlashAttention-2 |
Architecture Diagram
Evaluation Benchmarks
Rank
#130
| Benchmark | Score | Rank |
|---|---|---|
General Text | 1393 | 126 |
Intelligence Index | 0.17 | 199 |
Rankings
Overall Rank
#130
Coding Rank
-
About Step 3.5 Flash
Step 3.5 Flash is StepFun's most capable open-source foundation model. Engineered on a sparse Mixture of Experts architecture with 196B total parameters but only 11B active per token, it delivers frontier reasoning and agentic capabilities with exceptional efficiency. Features 256K context window, supports text and image input, and achieves 74.4% on SWE-bench Verified and 51.0% on Terminal-Bench 2.0. Optimized for local deployment on consumer hardware including Mac Studio M4 Max and high-end GPUs. Powered by 3-way Multi-Token Prediction (MTP-3) for 100-350 tok/s generation throughput.
Specifications
Attention
Attention Structure
Grouped-Query Attention
Attention Heads
64
Key-Value Heads
8
Attention Head Dimension
128
Position Embedding
ROPE
Sliding Window Attention
Yes
Sliding Window Size
512
Normalization
RMS Normalization
Activation Function
GELU
Dimensions
Base Precision
FP16
Hidden Dimension Size
4,096
Number of Layers
45
FFN Intermediate Size (Dense)
1,280
Multi-Token Prediction Heads
3
Tokenizer
Vocabulary Size
128,896
Mixture of Experts
Total Expert Parameters
11.0B
Number of Experts
288
Active Experts
8
FFN Intermediate Size (per Expert)
1,280
About Step 3.5
Step 3.5 is StepFun's flagship frontier reasoning model family. Built on sparse Mixture-of-Experts (MoE) architecture, Step 3.5 models deliver frontier-level intelligence for agentic, reasoning, and coding tasks. The Flash variant selectively activates only 11B of its 196B parameters per token, achieving the reasoning depth of top-tier proprietary models while maintaining exceptional efficiency. Features 256K context window, native function calling, and Multi-Token Prediction for high-throughput inference. Released under Apache 2.0 license.
Other Step 3.5 Models
- No related models available