This chapter concentrates on gradient-based meta-learning approaches, a significant category of algorithms designed to learn model initializations or update rules conducive to fast adaptation. The core idea is to optimize parameters $\theta$ such that a few gradient steps on a new task's support set yield good performance on its query set.

We will start with a detailed look at Model-Agnostic Meta-Learning (MAML), covering its optimization objective and the computational aspects involved. Subsequently, we analyze computationally efficient approximations like First-Order MAML (FOMAML) and Reptile, comparing their mechanisms and performance implications. Implicit MAML (iMAML) will be introduced as an alternative approach offering potential stability and memory benefits. We'll also discuss common issues like optimization stability and gradient variance, presenting techniques to mitigate them. Finally, we consider the specific challenges and strategies for scaling these gradient-based methods to large foundation models, culminating in a practical implementation exercise focused on FOMAML.