Building Scalable Data Warehouses
Chapter 1: MPP Architecture and Storage Internals
Massively Parallel Processing Fundamentals
Decoupling Compute and Storage
Columnar Storage Formats and Compression
Micro-partitioning and Metadata Management
Hands-on practice: Inspecting Storage Profiles
Chapter 2: Advanced Data Modeling at Scale
Dimensional Modeling Constraints in Big Data
Data Vault 2.0 Implementation Patterns
Handling Semi-Structured Data
Schema Evolution and Versioning
Hands-on practice: Designing a Data Vault
Chapter 3: High-Throughput Ingestion Pipelines
Change Data Capture Architectures
Idempotency in Data Pipelines
Micro-batch vs Streaming Ingestion
Handling Late Arriving Data
Hands-on practice: Building a CDC Pipeline
Chapter 4: Query Optimization and Performance Tuning
Analyzing Query Execution Plans
Partition Pruning and Clustering Keys
Join Strategies: Broadcast vs Shuffle
Materialized Views and Caching Layers
Hands-on practice: Tuning High-Latency Queries
Chapter 5: Governance, Security, and Observability
Role-Based Access Control Hierarchies
Dynamic Data Masking and Tokenization
Row-Level Security Implementation
Resource Monitoring and Cost Control
Hands-on practice: Configuring Security Policies
Decoupling Compute and Storage
Was this section helpful?
- BigQuery: A Serverless Data Warehouse at Scale, James Shute, Radek Vingralek, David C. Smith, Eric Schrock, Justin Levandoski, Jonathan Rosenfeld, Matthew A. S. Franklin, Stephen O'Kane, Jeremy C. Chen, Shailesh G. Lele, Ryan P. Johnson, Michael Carey, David S. C. Hsiao, Andrew C. Pavlo, 2021 Proceedings of the 2021 International Conference on Management of Data (SIGMOD '21) (ACM) DOI: 10.1145/3448016.3457155 - Presents the architecture of Google BigQuery, focusing on its serverless, decoupled design for massive-scale data analytics and its use of cloud object storage.
- Cloud-Native Data Warehousing, Thomas Neumann, Alfons Kemper, Eike Schiewer, Tobias Müller, Andreas Heindl, Viktor Leis, 2019 Proceedings of the VLDB Endowment, Volume 12, Issue 12, Vol. 12 (VLDB Endowment) DOI: 10.14778/3352063.3352077 - Discusses the architectural principles and challenges of building cloud-native data warehouses, including the fundamental shift towards the separation of compute and storage.
- Designing Data-Intensive Applications: The Big Ideas Behind Reliable, Scalable, and Maintainable Systems, Martin Kleppmann, 2017 (O'Reilly Media) - Provides a comprehensive explanation of distributed systems concepts, including architectural patterns, consistency models like MVCC, caching strategies, and data storage principles relevant to decoupled data warehouses.
© 2025 ApX Machine LearningEngineered with