Introduction to LLM Fine-Tuning
Chapter 1: Foundations of Model Customization
What is Fine-Tuning?
Pre-training vs. Fine-Tuning
When to Fine-Tune: An Analytical Framework
Overview of Fine-Tuning Strategies
The Role of Transfer Learning in LLMs
Setting Up Your Development Environment
Chapter 2: Data Preparation for Fine-Tuning
Sourcing and Selecting High-Quality Datasets
Instruction-Based vs. Conversational Data Formats
Data Cleaning and Preprocessing Techniques
Creating and Structuring Custom Datasets
Tokenization for Fine-Tuning
Hands-on Practical: Building a Fine-Tuning Dataset
Chapter 3: Full Parameter Fine-Tuning
The Mechanics of Full Fine-Tuning
Architectural Considerations for Full Fine-Tuning
Managing Computational Resources
Configuring Training Arguments and Hyperparameters
Monitoring Training: Loss and Metrics
Saving and Loading Fine-Tuned Models
Practice: Full Fine-Tuning on a Small-Scale Model
Chapter 4: Parameter-Efficient Fine-Tuning (PEFT)
Introduction to Parameter-Efficient Fine-Tuning
Low-Rank Adaptation (LoRA): Theory and Operation
Implementing LoRA with the PEFT Library
Quantization and its effect on Fine-Tuning (QLoRA)
Other PEFT Methods: A Brief Survey
Comparing PEFT and Full Fine-Tuning Trade-offs
Hands-on Practical: Fine-Tuning with LoRA
Chapter 5: Evaluation and Deployment
Defining Performance Metrics for Generative Tasks
Quantitative Evaluation: ROUGE, BLEU, and Perplexity
Qualitative Evaluation: Human-in-the-Loop Assessment
Building an Evaluation Pipeline
Strategies for Merging Adapters with the Base Model
Preparing Models for Inference
Practice: Evaluating a Fine-Tuned Model
What is Fine-Tuning?
Was this section helpful?
- Attention Is All You Need, Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, Illia Polosukhin, 2017 Advances in Neural Information Processing Systems (NIPS 2017) DOI: 10.48550/arXiv.1706.03762 - Introduces the Transformer architecture, which is a basis for modern large language models, including those that are fine-tuned.
- BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding, Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova, 2019 Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) DOI: 10.48550/arXiv.1810.04805 - Details the pre-training and fine-tuning paradigm for deep bidirectional Transformers, establishing a key method for adapting large language models.
- Natural Language Processing with Transformers, Lewis Tunstall, Leandro von Werra, Thomas Wolf, 2022 (O'Reilly Media) - A practical guide to using and fine-tuning Transformer models for various natural language processing tasks, offering concrete examples and best practices.
- Transfer Learning in Natural Language Processing: A Survey, Sebastian Ruder, Iain Stewart, Jeremy Howard, 2019 Proceedings of the Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) DOI: 10.18653/v1/D19-1587 - Provides a comprehensive overview of transfer learning techniques in natural language processing, including the pre-training and fine-tuning approach discussed.