Setting up the Development Environment

To effectively explore the concepts of generalization, overfitting, and the techniques we'll discuss throughout this course, you'll need a consistent and functional Python environment. This setup will allow you to run the code examples, experiment with different regularization and optimization methods, and complete the hands-on practical sessions, starting with visualizing overfitting later in this chapter.

We recommend using a recent version of Python, ideally Python 3.8 or later. Managing dependencies is important for reproducibility, and we suggest using either Conda (specifically Miniconda or Anaconda) or Python's built-in venv module combined with pip.

Core Libraries

The primary libraries we will rely on are:

1. PyTorch: Our main deep learning framework. It provides the necessary tools for building, training, and evaluating neural networks, including implementations of regularization layers and optimization algorithms.
2. NumPy: The fundamental package for numerical computation in Python. PyTorch integrates well with NumPy, and we'll use it for various data manipulations.
3. Matplotlib (and optionally Seaborn): Essential for plotting and visualizing data, including learning curves, weight distributions, and model predictions, which are indispensable for diagnosing model behavior.
4. Scikit-learn: Useful for supplementary tools like data splitting, performance metrics, and potentially simple baseline models.

Installation using Conda

If you're using Conda, you can create a dedicated environment and install the required packages. Open your terminal or Anaconda Prompt and run:

# Create a new environment (e.g., named 'dl-regopt') with Python 3.9
conda create -n dl-regopt python=3.9

# Activate the environment
conda activate dl-regopt

# Install PyTorch, torchvision, torchaudio (adjust for your OS/CUDA version if needed)
# Check the official PyTorch website (pytorch.org) for the latest command
# Example for CPU-only on Linux/macOS:
conda install pytorch torchvision torchaudio cpuonly -c pytorch

# Install other libraries
conda install numpy matplotlib scikit-learn seaborn jupyterlab

Note: For GPU support (highly recommended for faster training), visit the official PyTorch website (https://pytorch.org/) and select the appropriate installation command based on your operating system and CUDA version.

Installation using pip and venv

If you prefer using pip with a virtual environment:

# Create a directory for your project (if you haven't already)
mkdir deeplearning-course
cd deeplearning-course

# Create a virtual environment (e.g., named '.venv')
python -m venv .venv

# Activate the environment
# On macOS/Linux:
source .venv/bin/activate
# On Windows:
# .\.venv\Scripts\activate

# Upgrade pip
python -m pip install --upgrade pip

# Install PyTorch (adjust for your OS/CUDA version if needed)
# Check the official PyTorch website (pytorch.org) for the latest command
# Example for CPU-only:
pip install torch torchvision torchaudio

# Install other libraries
pip install numpy matplotlib scikit-learn seaborn jupyterlab

Verifying the Installation

Once the installation is complete, you can quickly verify that the core components are working. Start a Python interpreter or a Jupyter Notebook and try importing the libraries:

import torch
import numpy as np
import matplotlib.pyplot as plt
import sklearn

print(f"PyTorch Version: {torch.__version__}")
print(f"NumPy Version: {np.__version__}")
print(f"Scikit-learn Version: {sklearn.__version__}")

# Check if CUDA (GPU support) is available for PyTorch
if torch.cuda.is_available():
    print(f"CUDA available: {torch.cuda.is_available()}")
    print(f"CUDA version: {torch.version.cuda}")
    print(f"Device name: {torch.cuda.get_device_name(0)}")
else:
    print("CUDA not available. Running on CPU.")

# Test a simple PyTorch tensor operation
x = torch.rand(5, 3)
print("\nSample PyTorch Tensor:")
print(x)

If these commands execute without errors and display the versions and tensor output, your environment is ready. You now have the tools needed to follow along with the practical examples, starting with visualizing the effects of overfitting. Having this consistent setup ensures that the code behaves as expected as we explore different techniques to improve model generalization.