To begin fine-tuning models, a properly configured and reproducible environment is essential. This setup forms the foundation for practical exercises in model customization. It ensures that the code runs as expected and that specific library versions required for large language model operations can be managed. Necessary hardware considerations and the step-by-step installation of PyTorch and the Hugging Face ecosystem are detailed.
A typical LLM fine-tuning environment is built on several layers of software and hardware. At the base is the hardware, preferably a GPU, which provides the computational power. On top of that, we install a deep learning framework like PyTorch. Finally, we use specialized libraries from the Hugging Face ecosystem to simplify loading models, preparing data, and running the training process.
The relationship between the core software components used for fine-tuning. PyTorch provides the fundamental tensor operations, while the Hugging Face libraries offer high-level abstractions for models, data, and training acceleration.
Fine-tuning a large language model is a computationally intensive task. While it's technically possible to run on a CPU, the process would be impractically slow. For effective fine-tuning, a powerful NVIDIA GPU with sufficient Video RAM (VRAM) is highly recommended.
If you do not have access to a local NVIDIA GPU, cloud platforms like Google Colab, Kaggle Notebooks, or dedicated cloud GPU instances from AWS, GCP, or Azure are excellent alternatives.
Before installing any packages, it is a best practice to create an isolated virtual environment. This prevents conflicts with other projects or system-level Python packages. We recommend using Python 3.10 or newer.
Using Python's built-in venv module, create and activate a new environment with the following commands in your terminal:
# Create a virtual environment named 'llm-finetune-env'
python -m venv llm-finetune-env
# Activate the environment
# On macOS and Linux:
source llm-finetune-env/bin/activate
# On Windows:
.\llm-finetune-env\Scripts\activate
Once activated, your terminal prompt should change to indicate you are now working inside the llm-finetune-env environment.
The installation command for PyTorch depends on your operating system and your GPU's CUDA version. It is important to install the version of PyTorch that is compiled for the version of CUDA you have installed.
nvidia-smi in your terminal.pip or conda command for your system.For example, to install PyTorch with support for CUDA 12.1, the command is typically:
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121
After the installation completes, you can verify that PyTorch was installed correctly and can detect your GPU by running this short Python script.
import torch
if torch.cuda.is_available():
print(f"PyTorch version: {torch.__version__}")
print(f"CUDA available: {torch.cuda.is_available()}")
print(f"GPU: {torch.cuda.get_device_name(0)}")
else:
print("PyTorch was installed without CUDA support.")
A successful output confirms your setup is ready for GPU-accelerated training.
With PyTorch installed, the next step is to add the core libraries from the Hugging Face ecosystem. These libraries provide the tools to download models, manage datasets, and execute the fine-tuning process efficiently.
Install the required libraries using a single pip command:
pip install transformers datasets accelerate peft
Let's briefly review the role of each package:
transformers: Provides access to thousands of pre-trained models and their tokenizers, along with the Trainer API for simplifying the training loop.datasets: An efficient library for loading, processing, and caching large datasets, which is essential for managing the data used in fine-tuning.accelerate: A library that simplifies running PyTorch code across different hardware configurations (CPU, single GPU, multiple GPUs) with minimal code changes.peft: The Parameter-Efficient Fine-Tuning library, which contains implementations for methods like LoRA and QLoRA that we will use in Chapter 4.While many models and datasets on the Hugging Face Hub are public, some, like the Llama family of models, are "gated" and require you to accept terms of use before you can access them. Authenticating your environment with an access token allows your code to download these resources.
Once you have your token, run the following command in your terminal and paste the token when prompted:
huggingface-cli login
This command securely stores your token locally, allowing the libraries to automatically use it when required.
Your development environment is now fully configured. You have installed the necessary deep learning framework and the specialized libraries for adapting language models. With this setup in place, you are ready to move on to the next chapter, where we will begin the practical work of preparing a dataset for fine-tuning.
Cleaner syntax. Built-in debugging. Production-ready from day one.
Built for the AI systems behind ApX Machine Learning
Was this section helpful?
© 2026 ApX Machine LearningEngineered with