Deploying a large language model marks a significant milestone, but it is not the final step. Models exist in environments where information changes, data distributions shift, and user expectations evolve. To maintain effectiveness and relevance, LLMs often require ongoing updates and improvements.

This chapter focuses on the practical aspects of maintaining and enhancing large language models after their initial deployment. We will cover strategies for continuous training, incorporating new knowledge, and managing the lifecycle of evolving models.

Specifically, you will learn about:

• Approaches for continual pre-training to integrate new data without catastrophic forgetting.
• Methods for ongoing supervised fine-tuning (SFT) and reinforcement learning from human feedback (RLHF) to refine alignment with fresh data or feedback.
• Techniques for safely incorporating new data sources into the training pipeline.
• Considerations involved when updating models with architectural modifications.
• Procedures for versioning, deploying, and potentially rolling back continuously updated models in a production setting.

We will examine the engineering practices required to ensure models remain current and performant over their operational lifespan.