Building Scalable Data Warehouses
Chapter 1: MPP Architecture and Storage Internals
Massively Parallel Processing Fundamentals
Decoupling Compute and Storage
Columnar Storage Formats and Compression
Micro-partitioning and Metadata Management
Hands-on practice: Inspecting Storage Profiles
Chapter 2: Advanced Data Modeling at Scale
Dimensional Modeling Constraints in Big Data
Data Vault 2.0 Implementation Patterns
Handling Semi-Structured Data
Schema Evolution and Versioning
Hands-on practice: Designing a Data Vault
Chapter 3: High-Throughput Ingestion Pipelines
Change Data Capture Architectures
Idempotency in Data Pipelines
Micro-batch vs Streaming Ingestion
Handling Late Arriving Data
Hands-on practice: Building a CDC Pipeline
Chapter 4: Query Optimization and Performance Tuning
Analyzing Query Execution Plans
Partition Pruning and Clustering Keys
Join Strategies: Broadcast vs Shuffle
Materialized Views and Caching Layers
Hands-on practice: Tuning High-Latency Queries
Chapter 5: Governance, Security, and Observability
Role-Based Access Control Hierarchies
Dynamic Data Masking and Tokenization
Row-Level Security Implementation
Resource Monitoring and Cost Control
Hands-on practice: Configuring Security Policies
Massively Parallel Processing Fundamentals
Was this section helpful?
- The Snowflake Elastic Data Warehouse, Benoit Dageville, Allison Lee, Thuan Thai, et al., 2016 Proceedings of the 2016 ACM SIGMOD International Conference on Management of Data (ACM) DOI: 10.1145/2882903.2903730 - Describes the architecture of a prominent cloud-native MPP data warehouse, illustrating the separation of compute and storage and elastic scaling, directly addressing modern MPP developments.
- Validity of the Single Processor Approach to Achieving Large Scale Computing Capabilities, Gene M. Amdahl, 1967 AFIPS Conference Proceedings, Vol. 30 (Association for Computing Machinery) DOI: 10.1145/1465482.1465560 - The original paper introducing Amdahl's Law, a fundamental principle describing the theoretical speedup limitations of parallel processing due to sequential portions of tasks.
© 2025 ApX Machine LearningEngineered with