While previous chapters detailed various meta-learning algorithms, directly applying them to massive foundation models often encounters significant computational barriers. The scale of modern Large Language Models and Vision Transformers necessitates adaptation techniques that are both data-efficient for few-shot scenarios and computationally tractable. This chapter introduces practical strategies tailored for adapting these large pre-trained models, including methods that complement or serve as alternatives to the meta-learning approaches discussed earlier.

You will examine Parameter-Efficient Fine-Tuning (PEFT) methods, which focus on modifying only a small subset of the model's parameters during adaptation. Key techniques covered include:

• Adapter Modules: Injecting small, trainable modules into the layers of a fixed foundation model.
• Low-Rank Adaptation (LoRA): Adapting weight matrices by learning low-rank updates.
• Prompt Tuning and Prefix Tuning: Learning continuous embeddings prepended to input sequences or layer activations.

This chapter analyzes the mechanisms behind these techniques, compares their performance characteristics and computational requirements to meta-learning algorithms, and considers hybrid approaches. You will also gain hands-on experience by implementing LoRA to adapt a foundation model on a few-shot task.