Standard dimensional modeling places heavy emphasis on the end-user experience, prioritizing simplified query logic through denormalization. However, in a distributed MPP environment processing petabytes of data, the maintenance overhead of Type 2 Slowly Changing Dimensions (SCD) can become a bottleneck. Updating millions of rows to close distinct record versions triggers massive data shuffling across compute nodes. Data Vault 2.0 addresses this by decoupling business keys, relationships, and descriptive attributes into distinct structures. This separation enables 100% parallel loading, eliminates ingestion dependencies, and supports agile schema evolution without refactoring existing tables.
Data Vault 2.0 relies on three primary structures: Hubs, Links, and Satellites. Unlike the star schema which centers on a fact table, the Data Vault centers on the business key.
A Hub contains only the unique list of business keys for a core business entity (e.g., Customer, Product, Invoice). It does not contain descriptive data or relationships. The primary goal of the Hub is to establish a distinct list of identifiers that other structures can reference.
In traditional warehousing, surrogate keys are often sequential integers generated by the database. In MPP systems, generating sequential IDs prevents parallel loading because the loader must wait for the centralized counter. Data Vault 2.0 replaces sequential IDs with Hash Keys.
The Hash Key is a deterministic hash (typically MD5 or SHA-256) of the business key.
This allows multiple compute nodes to calculate keys independently without coordination, enabling linear scalability during ingestion.
Links represent the associations between Hubs. They resolve many-to-many relationships physically, regardless of the logical cardinality. A Link table contains its own Hash Key, the Hash Keys of the Hubs it connects, and the Load Date. By externalizing relationships into Links, you can add new relationships without altering the Hubs or existing Satellites.
Satellites store the descriptive attributes (context) and history. A Satellite is attached to either a Hub or a Link. It contains the parent's Hash Key, the Load Date, a Hash Diff (for change detection), and the actual data columns.
When a source record changes, you do not update the existing row. Instead, you insert a new row into the Satellite with a new Load Date. This "insert-only" architecture is highly efficient on log-structured merge trees and columnar storage engines common in Snowflake and BigQuery.
Structure of a basic Data Vault model showing the relationship between Hubs (blue), Links (red), and Satellites (green).
The scalability of Data Vault comes from its loading pattern. Because Hash Keys are deterministic, you can load Hubs, Links, and Satellites in parallel streams. The specific challenge in high-volume environments is Change Data Capture (CDC).
To avoid looking up the previous record value to detect changes (which requires an expensive self-join or window function), Data Vault uses a Hash Diff. The Hash Diff is calculated by concatenating all descriptive columns in the source feed and hashing the result.
During the staging process, you calculate the Hash Diff for incoming records. The loading logic becomes a simple set comparison:
HashDiff_Incoming.Hub_Hash_Key.Satellite.Hash_Key is NULL (new parent) OR Staging.HashDiff != Satellite.HashDiff (changed data).This comparison uses the MPP engine's ability to perform massive hash joins efficiently.
A common anti-pattern in early Data Vault implementations is grouping all attributes of a business entity into a single Satellite. In petabyte-scale warehouses, this leads to excessive I/O scanning.
Attributes often change at different velocities. For example, a customer's LoyaltyTier might change once a year, while their LastLoginDate changes daily. If stored in the same Satellite, every login event necessitates duplicating the static LoyaltyTier data in a new row, bloating storage and slowing down historical queries.
Split Satellites based on:
Sat_Cust_Salesforce, Sat_Cust_SAP). This prevents a delay in one source system from blocking the loading of another.While Data Vault excels at writing, the high degree of normalization requires many joins to reconstruct a "flat" view of the data. Joining a Hub to five different Satellites and two Links can degrade query latency.
To resolve this for the consumption layer, we implement Point-in-Time (PIT) tables. A PIT table is a bridge table that pre-calculates the valid joins for specific time snapshots.
A PIT table structure typically includes:
By computing these temporal alignments during the batch window (or continuously), downstream queries become simple equi-joins on primary keys, enabling the MPP optimizer to use partition pruning effectively.
Comparison of query latency scaling when joining multiple satellites directly versus using a pre-computed Point-in-Time (PIT) structure.
Schema drift is inevitable. In a Star Schema, adding a column usually requires an ALTER TABLE command on a potentially massive fact table, which can lock resources. In Data Vault, schema evolution is handled by adding new Satellites.
If a source system adds five new fields describing a customer:
ALTER TABLE, but generally safe in Snowflake/BigQuery as it's a metadata operation).Sat_Customer_v2) linked to the same Hub.Option B is preferred when the new data arrives from a distinct micro-service or has a different loading cadence. The Hub remains untouched. The Link remains untouched. Downstream views are updated to include the new columns, but the physical ingestion pipelines for existing data remain uninterrupted. This additive nature makes Data Vault particularly effective for agile engineering teams managing continuous integration pipelines.
Was this section helpful?
© 2026 ApX Machine LearningEngineered with