DeepSeek-V4-Flash-Vision-Exp

Active Parameters

290.9B

Context Length

1.05M

Modality

Text

Architecture

Mixture of Experts

KV Cache Rate

86 KB/tok

License

MIT License

Release Date

31 Aug 2026

Knowledge Cutoff

-

API Pricing (per 1M)

Input: $0.44 · Output: $1.32

Hardware & System Requirements

Precision

Native

Context

Workload Memory: 583.01 GB

FP16 · 8K context

Base Weights: 579.84 GB

KV Cache: 0.67 GB

Runtime & Graphs: 2.50 GB

Minimum Required Hardware

8x NVIDIA H200 141GB

583.0 / 1015.2 GB usable · TP 8

Determine requirements for your specific workload:

Upgrade to Pro

Runtime requirements

Checkpoint format

FP16

CUDA

>= 11.8

PyTorch

>= 2.1

Transformers

>= 5.0.0

ArchitecturePrecisionAttention kernel

Blackwell, Hopper

Native FP16

FlashMLA

Ada Lovelace, Ampere

Native FP16

FlashAttention-2

Architecture Diagram

Input TokensToken EmbeddingPosition: RoPEHidden: 4.1k · Context: 1.05M · Vocab: 129.3kx 43 layersRMSNormPre-AttentionDeepSeek Sparse Attention64Q / 1KV heads · SW: 128Head dim: 512+RMSNormPre-FFNSparse MoE FFN (6/256 experts)SwiGLUIntermediate: 2k+Final RMSNormOutput Logits

Evaluation Benchmarks

Rank

#36

BenchmarkScoreRank

Agentic Coding

LiveBench Agentic

0.65

5

0.79

12

Agentic Index

Artificial Analysis
max

0.47

23

0.77

25

LiveBench Average

LiveBench Average

0.77

26

0.85

35

0.88

42

max

0.65

59

0.68

60

Intelligence Index

Artificial Analysis
max

0.35

67

Rankings

Overall Rank

#36

Coding Rank

#52

About DeepSeek-V4-Flash-Vision-Exp

DeepSeek-V4-Flash-Vision-Exp is an experimental multimodal vision-language model based on DeepSeek's V4 architecture. It delivers fast visual reasoning, document parsing, and multimodal instruction following.

Specifications

Attention

Attention Structure

DeepSeek Sparse Attention

Attention Heads

64

Key-Value Heads

1

Attention Head Dimension

512

Position Embedding

ROPE

RoPE Theta

10,000

Sliding Window Attention

Yes

Sliding Window Size

128

Normalization

RMS Normalization

Activation Function

SwigLU

Dimensions

Base Precision

FP16

Hidden Dimension Size

4,096

Number of Layers

43

Multi-Token Prediction Heads

3

Tokenizer

Vocabulary Size

129,280

Mixture of Experts

Total Expert Parameters

20.4B

Number of Experts

256

Active Experts

6

Shared Experts

1

FFN Intermediate Size (per Expert)

2,048

About DeepSeek V4

DeepSeek-V4 is DeepSeek's latest generation of highly efficient Mixture-of-Experts language models, featuring a novel hybrid attention architecture combining Compressed Sparse Attention (CSA) and Heavily Compressed Attention (HCA) that dramatically improves long-context efficiency. Pre-trained on 32T+ tokens with a comprehensive post-training pipeline including domain-specific expert cultivation and unified model consolidation. Both V4-Pro and V4-Flash support 1M context length as standard, with three reasoning effort modes (Non-think, Think High, Think Max). Released open-source under MIT license on April 24, 2026.


Other DeepSeek V4 Models
DeepSeek-V4-Flash-Vision-Exp - VRAM Requirements, Specs & Benchmarks