You've learned how data is gathered and stored. Now, we focus on making that data useful. This chapter introduces the fundamental ways data is processed after collection.

You will learn about:

• The differences between batch processing, which handles large amounts of data at scheduled intervals, and stream processing, which deals with data in near real-time.
• A high-level view of processing frameworks and the compute resources they require.
• Basic techniques for data cleaning and data validation to improve data quality.

By the end of this chapter, you will understand the core methods used to transform raw data into formats suitable for analysis and other applications. We will also apply these ideas with a simple data cleaning exercise.