Lambda and Kappa Architectures

Was this section helpful?

References

Big Data: Principles and Best Practices of Scalable Realtime Data Systems, Nathan Marz and James Warren, 2015 (Manning Publications) - This foundational book introduces and comprehensively explains the Lambda Architecture.
Questioning the Lambda Architecture, Jay Kreps, 2014 (O'Reilly) - This influential article critically examines the Lambda Architecture and proposes the Kappa Architecture as a simplified alternative.
The Log: What every software engineer should know about real-time data's unifying abstraction, Jay Kreps, 2013 (Confluent Blog) - Discusses the distributed log as a unifying concept for real-time data systems, forming the theoretical basis for the Kappa Architecture.
Structured Streaming Programming Guide, Apache Spark Development Team, 2023 (Apache Software Foundation) - Official documentation for Apache Spark's Structured Streaming, demonstrating how a single API can handle both batch and stream processing, supporting the Kappa approach.
Delta Lake Documentation, The Delta Lake Project, 2023 (The Delta Lake Project) - Official documentation detailing Delta Lake's features, such as ACID transactions and upserts, which enable the 'Kappa Plus' architecture within data lakes.