State Backends: HashMap versus RocksDB

The choice of a state backend is the single most significant configuration decision when deploying Flink in production. It dictates not only where your data resides, on the Java Heap or on disk, but also fundamentally changes the performance profile, garbage collection behavior, and recovery mechanics of your streaming pipeline.

Flink provides two primary production-ready state backends: the HashMapStateBackend and the EmbeddedRocksDBStateBackend. While the DataStream API abstraction allows you to write code without worrying about storage specifics, the underlying physical execution varies drastically between these two options.

The HashMapStateBackend

The HashMapStateBackend holds data internally as objects on the Java Heap. When your application updates a ValueState<Integer>, Flink updates an Integer object directly in memory. This approach effectively uses a nested HashMap structure where the outer map is keyed by the KeyGroup and the inner map is keyed by the user-defined key.

Because state acts as native Java objects, access times are extremely fast. Reading or writing state requires no serialization or deserialization (SerDe) during processing. The operation is simply a pointer dereference, providing $O(1)$ access complexity with negligible overhead.

However, this performance comes with strict boundary conditions. Since the state lives on the JVM Heap, the total size of your state plus the memory required for processing must fit within the configured heap size.

Memory Layout and Garbage Collection

In high-throughput scenarios, the HashMapStateBackend exerts significant pressure on the Garbage Collector (GC). As state objects are created, modified, and discarded (for example, in windowing operations), the Old Generation of the heap can fill up, triggering full GC pauses. These stop-the-world events halt processing, causing latency spikes that can violate strict Service Level Agreements (SLAs).

The following diagram illustrates the memory layout within a TaskManager when using the HashMap backend versus the RocksDB backend.

The architectural difference between storing state as heap objects versus off-heap serialized bytes affects both latency and scalability.

The EmbeddedRocksDBStateBackend

For applications requiring massive state, spanning hundreds of gigabytes or terabytes, relying on the Java Heap is infeasible. The EmbeddedRocksDBStateBackend addresses this by storing state in a locally running RocksDB instance. RocksDB is a log-structured merge-tree (LSM tree) key-value store optimized for fast writes.

When using this backend, state is stored in off-heap native memory and spilled to the local disk (ideally SSDs) when memory buffers are full. This architecture decouples state size from JVM heap size. You can support state sizes limited only by the available disk space on your cluster nodes.

The Serialization Tax

The trade-off for this scalability is the cost of serialization. Unlike the HashMap backend, every read or write operation in RocksDB requires data to cross the JNI (Java Native Interface) boundary.

1. Write: Java Object $\rightarrow$ Serialized Byte Array $\rightarrow$ JNI $\rightarrow$ RocksDB MemTable.
2. Read: RocksDB $\rightarrow$ JNI $\rightarrow$ Byte Array $\rightarrow$ Deserialized Java Object.

This serialization overhead introduces latency. While usually in the range of microseconds, it accumulates in pipelines performing millions of state accesses per second. The total cost of a state access operation $C_{total}$ can be approximated as:

$C_{total} = C_{compute} + C_{serialization} + C_{JNI} + C_{disk\_io}$

Where $C_{disk\_io}$ becomes non-zero only when the requested data is not found in the RocksDB block cache and must be fetched from the disk.

Comparative Performance Analysis

Choosing between these backends requires analyzing the intersection of your latency requirements and state volume.

• HashMap: Offers the lowest latency but hits a "cliff" when state size approaches heap limits, leading to OutOfMemoryError or massive GC thrashing.
• RocksDB: Provides predictable, consistent performance regardless of state size, provided the local disk I/O throughput is sufficient. It avoids Java GC pressure on the state itself, as the data resides in native memory.

The chart below projects the relationship between state size and processing latency for both backends.

The HashMap backend maintains low latency until garbage collection saturation, whereas RocksDB maintains stability as state grows.

Snapshot Strategies and Recovery

The choice of backend also impacts how checkpoints (snapshots of the application state) are created. Checkpointing is the mechanism Flink uses to guarantee fault tolerance.

HashMap Snapshots: Historically, this backend required a copy-on-write mechanism that could be heavy on memory during the snapshot phase. While Flink supports asynchronous snapshots for the HashMap backend, the state must be serialized at the moment of the checkpoint. If the state is large, serializing the entire heap to the distributed file system (like S3 or HDFS) consumes significant network bandwidth and CPU.

RocksDB Incremental Checkpoints: A distinct advantage of the RocksDB backend is its support for incremental checkpoints. Since RocksDB immutable data is stored in SSTables (Sorted String Tables) on disk, Flink can track which files have changed since the last checkpoint. During a snapshot, Flink uploads only the new or modified SSTables to the checkpoint storage rather than the entire state.

This feature is critical for pipelines with large state (e.g., 5TB) where the daily "churn" (changed state) might only be 50GB. Incremental checkpointing dramatically reduces the duration of the checkpoint process and the storage costs associated with maintaining history.

Selection Criteria Matrix

When architecting your pipeline, apply the following criteria to select the appropriate backend.

1. State Size: If your estimated state is $< 10$ GB per slot, HashMapStateBackend is generally safe. Above 20-30 GB per slot, EmbeddedRocksDBStateBackend is necessary to avoid GC issues.
2. Latency Sensitivity: If you require sub-millisecond latency for complex calculations and your state fits in memory, use HashMap. RocksDB will introduce serialization overhead.
3. Interaction Patterns: If your logic frequently updates the same keys (hot keys), HashMap performs better as it avoids repeated serialization.
4. Availability: If you need fast checkpoints for a large state to ensure tight Recovery Point Objectives (RPO), RocksDB with incremental checkpointing is the superior choice.

To configure the backend explicitly in your application code, use the StreamExecutionEnvironment. However, it is standard practice to define this in the flink-conf.yaml file to decouple infrastructure decisions from application logic.

// Example of setting the state backend programmatically (Not recommended for production flexibility)
env.setStateBackend(new EmbeddedRocksDBStateBackend(true)); // true enables incremental checkpointing

In the next section, we will examine the mechanics of Asynchronous Barrier Snapshots to understand how Flink coordinates these state backends to achieve consistent distributed state without halting the data stream.