Building Scalable Data Warehouses
Chapter 1: MPP Architecture and Storage Internals
Massively Parallel Processing Fundamentals
Decoupling Compute and Storage
Columnar Storage Formats and Compression
Micro-partitioning and Metadata Management
Hands-on practice: Inspecting Storage Profiles
Chapter 2: Advanced Data Modeling at Scale
Dimensional Modeling Constraints in Big Data
Data Vault 2.0 Implementation Patterns
Handling Semi-Structured Data
Schema Evolution and Versioning
Hands-on practice: Designing a Data Vault
Chapter 3: High-Throughput Ingestion Pipelines
Change Data Capture Architectures
Idempotency in Data Pipelines
Micro-batch vs Streaming Ingestion
Handling Late Arriving Data
Hands-on practice: Building a CDC Pipeline
Chapter 4: Query Optimization and Performance Tuning
Analyzing Query Execution Plans
Partition Pruning and Clustering Keys
Join Strategies: Broadcast vs Shuffle
Materialized Views and Caching Layers
Hands-on practice: Tuning High-Latency Queries
Chapter 5: Governance, Security, and Observability
Role-Based Access Control Hierarchies
Dynamic Data Masking and Tokenization
Row-Level Security Implementation
Resource Monitoring and Cost Control
Hands-on practice: Configuring Security Policies
Hands-on practice: Inspecting Storage Profiles
Was this section helpful?
- Clustering Keys and Clustered Tables, Snowflake Documentation, 2024 (Snowflake) - Explains Snowflake's automatic clustering, micro-partitions, and the SYSTEM$CLUSTERING_INFORMATION function.
- Designing Data-Intensive Applications: The Big Ideas Behind Reliable, Scalable, and Maintainable Systems, Martin Kleppmann, 2017 (O'Reilly Media) - A comprehensive guide covering data storage, indexing, partitioning strategies, and distributed systems, relevant to optimizing data warehouse storage.
- C-Store: A Column-Oriented DBMS, Michael Stonebraker, Daniel J. Abadi, Adam Batkin, Xuedong Chen, Mitch Cherniack, Miguel Ferreira, Edmond Lau, Amerson Lin, Samuel R. Madden, Elizabeth J. O'Neil, Patrick E. O'Neil, Alexander Rasin, Nga Tran, Stan B. Zdonik, 2005 Proceedings of the 31st International Conference on Very Large Data Bases (VLDB) (VLDB Endowment) - Presents a foundational architecture for column-oriented database management systems, detailing storage structures and query processing essential for understanding modern MPP data warehouses.
- Introduction to BigQuery storage, Google Cloud Documentation, 2024 (Google Cloud) - Describes BigQuery's columnar storage format, physical layout, and how partitioning and clustering enhance query performance.
© 2025 ApX Machine LearningEngineered with