ApX logoApX logo
ModelsCalculatorHardwareCoursesPricing
ApX logoApX logo
ModelsCalculatorHardwareCoursesPricing

All Courses

Fine-tuning and Adapting Large Language Models
Chapter 1: Foundations of LLM Adaptation
Recap: Pre-trained Language Models and Transformers
The Need for Fine-tuning and Adaptation
Transfer Learning Paradigms in NLP
Architectural Considerations for Fine-tuning
Overview of Fine-tuning Approaches
Chapter 2: Data Preparation for Fine-tuning
Instruction Tuning Principles
Sourcing and Constructing Instruction Datasets
Formatting Data for Supervised Fine-tuning (SFT)
Domain Adaptation Data Requirements
Handling Data Scarcity and Imbalance
Data Augmentation Techniques for Text
Practice: Preparing an Instruction Tuning Dataset
Chapter 3: Full Parameter Fine-tuning
Mechanism of Full Fine-tuning
Setting up the Training Loop
Hyperparameter Tuning Strategies
Regularization Techniques to Prevent Overfitting
Managing Computational Resources
Checkpointing and Resuming Training
Hands-on Practical: Full Fine-tuning a Smaller LLM
Chapter 4: Parameter-Efficient Fine-tuning (PEFT) Methods
Rationale for Parameter Efficiency
Low-Rank Adaptation (LoRA)
Quantized Low-Rank Adaptation (QLoRA)
Adapter Modules
Prompt Tuning
Prefix Tuning
Comparison of PEFT Techniques
Implementation with Hugging Face PEFT Library
Hands-on Practical: Fine-tuning with LoRA
Hands-on Practical: Fine-tuning with QLoRA
Chapter 5: Advanced Fine-tuning Strategies
Multi-Task Fine-tuning
Sequential Adaptation and Continual Learning
Mitigating Catastrophic Forgetting
Introduction to Reinforcement Learning from Human Feedback (RLHF)
Reward Model Training
Policy Optimization with PPO
Challenges in Advanced Adaptation
Chapter 6: Evaluation and Analysis of Fine-tuned Models
Limitations of Standard NLP Metrics
Evaluating Instruction Following Capabilities
Assessing Factual Accuracy and Hallucinations
Bias and Fairness Assessment Techniques
Robustness Evaluation (Adversarial Attacks, OOD)
Model Calibration Assessment
Qualitative Analysis and Error Categorization
Human Evaluation Protocols
Practice: Analyzing Model Outputs for Errors
Chapter 7: Optimization and Deployment Considerations
Memory Optimization during Training
Accelerating Training with Distributed Strategies
Post-tuning Optimization: Quantization
Post-tuning Optimization: Pruning
Merging PEFT Adapters
Model Serialization and Packaging
Inference Serving Frameworks
Monitoring Fine-tuned Models in Production

Model Serialization and Packaging

Saving and packaging fine-tuned models is essential for reliable deployment and sharing, especially after optimization techniques like quantization or pruning have been applied, or PEFT adapters have been merged. Proper serialization ensures that the model's learned parameters, configuration, and associated components like tokenizers are stored correctly and can be loaded consistently in different environments. Packaging bundles the model with its dependencies and documentation, making management and deployment simpler.

Standard Model Saving Formats

While various serialization formats exist, the ecosystem around LLMs has largely converged on a few common practices, primarily driven by libraries like Hugging Face's transformers.

Hugging Face save_pretrained / from_pretrained

This is arguably the most common method for saving and loading transformer-based models, including fine-tuned LLMs. When you call model.save_pretrained(save_directory), the library typically saves several components:

  • Model Weights: The learned parameters of the model. These are often saved in PyTorch's .bin format (pytorch_model.bin) or increasingly in the safetensors format (model.safetensors).
  • Model Configuration: A JSON file (config.json) storing the model's architecture, hyperparameters (like hidden size, number of layers), and other metadata necessary to reconstruct the model structure.
  • Tokenizer Files: The vocabulary file(s) (e.g., vocab.json, merges.txt, sentencepiece.model) and the tokenizer configuration (tokenizer_config.json, special_tokens_map.json) required to correctly process text input for the model.
# Example: Saving a model and tokenizer
from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "gpt2" # Or your fine-tuned model path
model = AutoModelForCausalLM.from_pretrained(model_name)
tokenizer = AutoTokenizer.from_pretrained(model_name)

# ... fine-tuning or optimization happens here ...

output_dir = "./my_finetuned_model"
model.save_pretrained(output_dir)
tokenizer.save_pretrained(output_dir)

print(f"Model and tokenizer saved to {output_dir}")

# Later, loading the model
loaded_model = AutoModelForCausalLM.from_pretrained(output_dir)
loaded_tokenizer = AutoTokenizer.from_pretrained(output_dir)
print("Model and tokenizer loaded successfully.")

Using save_pretrained ensures that all necessary components are saved in a standardized structure, facilitating easy loading via from_pretrained.

Safetensors

Safetensors (.safetensors) is a secure and fast file format for storing tensors, designed as an alternative to Python's pickle format, which is known to have security vulnerabilities (it can execute arbitrary code). Main advantages include:

  1. Security: Loading safetensors files is safe as it doesn't involve arbitrary code execution.
  2. Speed: Loading tensors is often faster, especially for large models, as metadata is stored separately, allowing for efficient memory mapping.
  3. Lazy Loading: The format facilitates loading only parts of a model into memory if needed, although full model loading is more common.
  4. Cross-Platform Compatibility: It provides a consistent format across different frameworks and languages.

Hugging Face libraries increasingly support and default to safetensors. You can explicitly request it:

# Saving with safetensors explicitly
model.save_pretrained(output_dir, safe_serialization=True)

If a model.safetensors file exists in the directory, from_pretrained will prioritize loading it over pytorch_model.bin.

ONNX (Open Neural Network Exchange)

ONNX provides a standardized format for representing machine learning models. Exporting your fine-tuned LLM to ONNX can offer benefits like:

  • Interoperability: Run the model using various ONNX-compatible runtimes (like ONNX Runtime) on different platforms (CPU, GPU, specialized hardware).
  • Potential Performance Gains: ONNX runtimes often incorporate graph optimizations that can accelerate inference.

Exporting usually involves tracing the model with sample inputs. Libraries like transformers.onnx provide utilities to simplify this process.

# Example of exporting to ONNX
from pathlib import Path
from transformers.onnx import export, FeaturesManager

# Assume 'model' and 'tokenizer' are loaded
onnx_output_dir = Path("./my_finetuned_model_onnx")
onnx_output_dir.mkdir(parents=True, exist_ok=True)

# Get the feature (e.g., 'causal-lm') and configuration
model_kind, model_onnx_config = FeaturesManager.check_supported_model_or_raise(model, feature='causal-lm')
onnx_config = model_onnx_config(model.config)

# Export the model
onnx_inputs, onnx_outputs = export(
    tokenizer=tokenizer,
    model=model,
    config=onnx_config,
    output=onnx_output_dir / "model.onnx",
    opset=onnx_config.default_onnx_opset # e.g., 13 or higher
)
print(f"Model exported to ONNX format at {onnx_output_dir}")

Challenges can arise with dynamic shapes, custom operations, or ensuring compatibility with the target ONNX runtime version. Thorough testing after export is necessary.

Handling PEFT Adapters

If you used Parameter-Efficient Fine-Tuning (PEFT) methods like LoRA or QLoRA, serialization requires special attention because you typically only saved the adapter weights, not the entire model.

  • Saving Adapters: PEFT libraries usually have their own saving mechanisms. For Hugging Face's peft, you save the adapter configuration and weights:

    # Assuming 'peft_model' is your model with adapters
    adapter_output_dir = "./my_lora_adapter"
    peft_model.save_pretrained(adapter_output_dir)
    print(f"Adapter saved to {adapter_output_dir}")
    # This saves adapter_model.bin (or .safetensors) and adapter_config.json
    
  • Loading Adapters: To use the adapter, you first load the base pre-trained model and then load the adapter weights onto it:

    from peft import PeftModel, PeftConfig
    from transformers import AutoModelForCausalLM, AutoTokenizer
    
    base_model_name = "gpt2" # The original base model
    adapter_dir = "./my_lora_adapter"
    
    # Load base model and tokenizer
    base_model = AutoModelForCausalLM.from_pretrained(base_model_name)
    tokenizer = AutoTokenizer.from_pretrained(base_model_name) # Often use base tokenizer
    
    # Load the adapter onto the base model
    inference_model = PeftModel.from_pretrained(base_model, adapter_dir)
    inference_model.eval() # Set to evaluation mode
    print("Base model loaded with PEFT adapter.")
    

    This approach keeps the base model weights separate, allowing multiple adapters to be potentially used with the same base model instance.

  • Merging and Saving: As discussed in the previous section ("Merging PEFT Adapters"), you can merge the adapter weights into the base model's weights. After merging, you obtain a standard transformers model object. You can then save this merged model using the standard model.save_pretrained(output_dir) method. This simplifies deployment as you only need to manage one set of model artifacts, but you lose the modularity of separate adapters.

Packaging Best Practices

Saving the model weights and configuration is only part of the story. Proper packaging involves bundling everything required for the model to run correctly and be understood by others.

  • Dependencies: Explicitly list all required Python libraries and their versions in a requirements.txt file or an environment specification file (like Conda's environment.yml). This includes libraries like torch, transformers, peft, sentencepiece, protobuf (for ONNX), etc. Inconsistent dependency versions are a frequent source of errors.
  • Configuration Files: Ensure all necessary configuration files (config.json, tokenizer_config.json, etc.) generated during save_pretrained are included alongside the model weights.
  • Model Card: Include a README.md file formatted as a Model Card. This file should document:
    • Model details (base model, fine-tuning task).
    • Fine-tuning data description.
    • Intended uses and limitations.
    • Evaluation results (metrics, bias assessment).
    • Ethical implications and potential risks.
    • Example usage code. Hugging Face Hub provides a standard template for model cards.
  • Containerization (Docker): For deployment, package the model artifacts, inference code, dependencies, and necessary system libraries into a Docker container. A Dockerfile defines the steps to build this image. This creates a self-contained, reproducible environment that simplifies deployment across different systems.

Components typically included when packaging a fine-tuned LLM for deployment. PEFT adapters might be included separately or merged into the main model weights. Containerization provides an isolated deployment environment.

By carefully serializing your optimized model and packaging it with its dependencies and documentation, you create a reliable and portable artifact ready for integration into downstream applications or sharing within the community. This systematic approach minimizes deployment friction and ensures reproducibility.

  • Hugging Face Transformers: Share and Use Models, Hugging Face, 2024 (Hugging Face) - A comprehensive guide to saving and loading models, configurations, and tokenizers using the save_pretrained and from_pretrained methods within the Hugging Face Transformers library.
  • safetensors GitHub Repository, Hugging Face, 2022 - The official repository providing technical specifications, implementations, and background on the safetensors format, emphasizing its security and performance advantages over traditional serialization methods.
  • Hugging Face PEFT Documentation: Package Structure, Hugging Face, 2024 (Hugging Face) - Documentation describing the specific procedures for saving, loading, and integrating Parameter-Efficient Fine-Tuning (PEFT) adapters, such as LoRA, with base language models.
  • ONNX (Open Neural Network Exchange) Documentation, ONNX Community, 2024 (Linux Foundation AI & Data (LFAI)) - The official resource for understanding the ONNX format, its ecosystem, and how it enables model interoperability and optimized inference across various hardware and runtimes.

© 2026 ApX Machine Learning