Chapter 4: Automation and CI/CD for ML

Manually executing the steps of the machine learning lifecycle is inefficient and susceptible to error. To build systems that are both scalable and reliable, we must introduce automation. This chapter adapts the established software engineering practices of Continuous Integration and Continuous Delivery (CI/CD) to the specific requirements of machine learning.

We will begin with the fundamentals of CI/CD and then examine how these practices are modified for ML. You will learn to differentiate between:

Continuous Integration (CI): Automating the testing of code and components.
Continuous Delivery (CD): Automating the release of a validated model.
Continuous Training (CT): A process specific to MLOps for automatically retraining models on new data.

The chapter explains how these elements combine to form an automated ML pipeline. We will also introduce orchestration tools that manage these workflows. The chapter concludes with a hands-on practical where you will build a simple CI pipeline using GitHub Actions to solidify your understanding.

Sections

4.1 Introduction to CI/CD
4.2 Continuous Integration (CI) for ML Code
4.3 Continuous Delivery (CD) for Models
4.4 Continuous Training (CT)
4.5 Building a Basic ML Pipeline
4.6 Introduction to Orchestration Tools
4.7 Hands-on Practical: Create a Simple CI Pipeline with GitHub Actions