Investigating Catastrophic Forgetting

Catastrophic forgetting (CF) is a well-known phenomenon in neural networks where a model trained sequentially on multiple tasks tends to abruptly lose performance on previously learned tasks after learning a new one. This occurs because the parameter updates required for the new task overwrite the parameters essential for remembering the old tasks. Large language models (LLMs), which are pre-trained on immense amounts of general data, face a significant concern of forgetting foundational knowledge during fine-tuning for a specific downstream task. Full fine-tuning, which updates all model parameters, is particularly susceptible to this problem. Understanding how well various PEFT methods mitigate catastrophic forgetting compared to full fine-tuning is a primary aspect of their overall assessment.

Why PEFT is Expected to Mitigate Forgetting

Parameter-Efficient Fine-Tuning methods were designed, in part, to address the computational burdens of full fine-tuning, but their architecture inherently offers a potential defense against catastrophic forgetting. The primary reasons include:

1. Frozen Base Model Parameters: Most PEFT techniques (LoRA, Adapters, Prefix/Prompt Tuning) keep the overwhelming majority of the original pre-trained model weights frozen. Fine-tuning only affects a small number of additional or modified parameters. Since the core knowledge resides within the millions or billions of frozen parameters, it's less likely to be erased by updates focused on the small set of tunable parameters.
2. Task-Specific Parameter Isolation: The newly introduced parameters (like LoRA matrices A and B, or adapter layers) are optimized specifically for the downstream task. This isolates the task-specific adaptation from the general knowledge embedded in the base model weights. In methods like LoRA, the update $\Delta W = BA$ is added to the original weight $W_0$, separating the original function from the adaptation.
3. Limited Update Capacity: The low dimensionality of the updates in methods like LoRA (controlled by rank $r$) or the bottleneck dimension in adapters inherently limits the capacity of the fine-tuning process to induce drastic changes across the entire model's behavior. While sufficient for adapting to a new task, this limited capacity might be insufficient to completely overwrite complex, distributed representations learned during pre-training.

Measuring Catastrophic Forgetting

To quantitatively assess how well a PEFT method preserves prior knowledge, we need systematic evaluation procedures. Common approaches include:

1. Sequential Task Fine-Tuning:

   • Step 1: Establish a baseline performance on a set of evaluation tasks (Task Set A) using the original pre-trained model. These tasks might cover general capabilities like commonsense reasoning, reading comprehension, or general knowledge.
   • Step 2: Fine-tune the model on a specific downstream task (Task B) using the chosen PEFT method (e.g., LoRA, QLoRA, Adapters).
   • Step 3: After fine-tuning on Task B, re-evaluate the model's performance on the original Task Set A.
   • Analysis: The difference in performance on Task Set A before and after fine-tuning on Task B indicates the degree of catastrophic forgetting. Compare this difference across various PEFT methods and against full fine-tuning.

2. Performance on General Benchmarks:

   • Fine-tune the model on the target task (Task B) using PEFT.
   • Evaluate the fine-tuned model not only on Task B but also on a broad suite of standard NLP benchmarks (e.g., GLUE, SuperGLUE, MMLU) that represent general language understanding capabilities.
   • Compare this performance profile to that of the original base model and a fully fine-tuned model. Significant degradation on these general benchmarks points towards forgetting.

Metrics: Standard performance metrics relevant to the evaluation tasks (Task Set A or general benchmarks) are used, such as accuracy, F1-score, perplexity, BLEU/ROUGE scores, etc. The primary measure of forgetting is the performance drop on these tasks after fine-tuning on the new task.

Empirical Observations and Influencing Factors

Studies comparing PEFT methods to full fine-tuning consistently demonstrate that PEFT significantly reduces catastrophic forgetting. While full fine-tuning might achieve slightly higher performance on the target task (Task B) in some cases, it often comes at the cost of substantial performance degradation on other tasks. PEFT methods typically strike a better balance, achieving strong performance on Task B while preserving much of the model's general capabilities.

Illustration of performance drop on a general knowledge benchmark (e.g., MMLU average accuracy) after fine-tuning on a specialized task (e.g., legal document analysis). PEFT methods demonstrate considerably less forgetting than full fine-tuning.

However, the degree of forgetting mitigation with PEFT is not absolute and can be influenced by several factors:

• PEFT Method: Different PEFT methods might exhibit varying levels of forgetting. For instance, the specific placement and design of adapters could matter.
• Tunable Parameter Count/Capacity: Increasing the rank $r$ in LoRA or the bottleneck dimension in adapters increases the capacity for adaptation but might also slightly increase the risk of forgetting if the capacity becomes large enough to interfere with base model representations.
• Task Similarity: Fine-tuning on a task very dissimilar to the pre-training data might lead to more noticeable forgetting than fine-tuning on a closely related task.
• Training Data Size and Duration: Extensive fine-tuning on a large dataset for many epochs, even with PEFT, could potentially lead to some degradation of general knowledge.

Limitations and Challenges

It's important to recognize that PEFT reduces, but doesn't necessarily eliminate, catastrophic forgetting entirely. Some performance degradation on unrelated tasks might still occur. Furthermore, there's often a trade-off: completely preventing any forgetting might limit the model's ability to fully adapt and achieve optimal performance on the new target task. The goal is typically significant mitigation rather than absolute prevention.

Research continues to explore ways to further enhance knowledge preservation within PEFT frameworks, sometimes drawing inspiration from continual learning techniques developed for standard training paradigms.

In summary, assessing the degree of catastrophic forgetting is a significant part of evaluating any fine-tuning strategy. PEFT methods generally offer a substantial advantage over full fine-tuning in preserving the valuable knowledge encoded within large pre-trained models, making them a more reliable choice for adapting LLMs across multiple tasks or domains over time. When selecting and configuring a PEFT approach, considering its potential impact on the model's pre-existing capabilities is essential for reliable deployment.