Hands-on practical: Develop an ML App with Compose

Okay, let's put the concepts from this chapter into practice. We will build a simple multi-container Machine Learning application using Docker Compose. Our application will consist of two services:

1. Inference API: A simple web service (using Flask) that loads a dummy model and serves predictions.
2. Redis Cache: A Redis instance used by the API to cache prediction results, demonstrating inter-container communication.

This setup mirrors common real-world scenarios where an ML service might depend on other backend components like databases or caches.

Project Setup

First, create a directory for our project, let's call it compose_ml_app. Inside this directory, create the following structure:

compose_ml_app/
├── app/
│   ├── __init__.py
│   ├── main.py
│   └── requirements.txt
├── Dockerfile
└── docker-compose.yml

• app/: This directory will contain our Flask API code.
• app/main.py: The main Python script for the Flask application.
• app/requirements.txt: Lists the Python dependencies for the API.
• Dockerfile: Defines how to build the image for our Inference API service.
• docker-compose.yml: Defines the multi-container application stack.

Building the Inference API

Let's create the components for our Flask API service.

1. app/requirements.txt: List the necessary Python libraries. For this example, we need Flask to create the API and redis-py to interact with the Redis cache.

   Flask==2.3.3
   redis==5.0.1
   

   Note: Using specific versions enhances reproducibility.

2. app/main.py: This script sets up a basic Flask app with one endpoint /predict. It simulates loading a model and making a prediction. Before predicting, it checks if the result for the given input data is already in the Redis cache. If so, it returns the cached result; otherwise, it computes the result, stores it in the cache, and then returns it.

   import os
   import time
   import redis
   from flask import Flask, request, jsonify
   
   app = Flask(__name__)
   
   # Connect to Redis - use the service name 'redis' defined in docker-compose.yml
   # Docker Compose automatically provides DNS resolution between services on the same network.
   redis_host = os.environ.get('REDIS_HOST', 'localhost')
   redis_port = int(os.environ.get('REDIS_PORT', 6379))
   cache = redis.StrictRedis(host=redis_host, port=redis_port, db=0, decode_responses=True)
   
   # Simulate loading a model (replace with actual model loading if needed)
   def load_model():
       print("Simulating model loading...")
       time.sleep(2) # Simulate time taken to load
       print("Model loaded.")
       # In a real app, you'd load your scikit-learn, TensorFlow, PyTorch model here
       return "dummy_model"
   
   model = load_model()
   
   @app.route('/predict', methods=['POST'])
   def predict():
       try:
           data = request.get_json()
           if not data or 'input' not in data:
               return jsonify({"error": "Invalid input. Expecting JSON with 'input' key."}), 400
   
           input_data = str(data['input']) # Use input as cache key
   
           # Check cache first
           cached_result = cache.get(input_data)
           if cached_result:
               print(f"Cache hit for input: {input_data}")
               return jsonify({"prediction": cached_result, "source": "cache"})
   
           # If not in cache, simulate prediction
           print(f"Cache miss for input: {input_data}. Predicting...")
           # Replace this with your actual model.predict() call
           time.sleep(0.5) # Simulate prediction time
           prediction_result = f"prediction_for_{input_data}"
   
           # Store result in cache with an expiration time (e.g., 1 hour)
           cache.setex(input_data, 3600, prediction_result)
           print(f"Stored prediction in cache for input: {input_data}")
   
           return jsonify({"prediction": prediction_result, "source": "computation"})
   
       except redis.exceptions.ConnectionError as e:
            print(f"Redis connection error: {e}")
            return jsonify({"error": "Could not connect to cache service."}), 503
       except Exception as e:
           print(f"An error occurred: {e}")
           return jsonify({"error": "An internal error occurred."}), 500
   
   @app.route('/health', methods=['GET'])
   def health_check():
       # Basic health check
       try:
           # Check connection to Redis cache
            cache.ping()
            return jsonify({"status": "ok", "cache_connected": True}), 200
       except redis.exceptions.ConnectionError:
            return jsonify({"status": "degraded", "cache_connected": False}), 503
   
   
   if __name__ == '__main__':
       app.run(host='0.0.0.0', port=5000)
   

   Notice how the Redis host is retrieved from the environment variable REDIS_HOST. We will set this in our docker-compose.yml file. The default is localhost for potential local testing outside Docker. We also added a basic /health endpoint.

3. app/__init__.py: Create an empty __init__.py file inside the app directory to make it a Python package. You can do this with touch app/__init__.py on Linux/macOS or create an empty file named __init__.py in Windows.

Creating the Dockerfile for the API

Now, define the Dockerfile in the project root (compose_ml_app/) to containerize the Flask application.

# Use an official Python runtime as a parent image
FROM python:3.9-slim

# Set the working directory in the container
WORKDIR /app

# Copy the requirements file into the container at /app
COPY app/requirements.txt .

# Install any needed packages specified in requirements.txt
# Use --no-cache-dir to reduce image size
RUN pip install --no-cache-dir -r requirements.txt

# Copy the application code into the container at /app
COPY ./app /app

# Make port 5000 available to the world outside this container
EXPOSE 5000

# Define environment variable defaults (can be overridden by docker-compose)
ENV REDIS_HOST=redis
ENV REDIS_PORT=6379

# Run main.py when the container launches
CMD ["python", "main.py"]

This Dockerfile sets up a Python 3.9 environment, installs dependencies, copies the application code, exposes the port Flask will run on, sets default environment variables for Redis connection, and specifies the command to start the application.

Defining the Multi-Container Application with docker-compose.yml

This is where Docker Compose comes in. Create the docker-compose.yml file in the project root (compose_ml_app/).

version: '3.8' # Specify compose file version

services:
  # Service 1: The Inference API
  api:
    build: . # Build the image from the Dockerfile in the current directory
    ports:
      - "5001:5000" # Map host port 5001 to container port 5000
    environment:
      - REDIS_HOST=redis # Override default, ensuring it uses the redis service name
      - REDIS_PORT=6379
    volumes:
      # Mount the local 'app' directory to '/app' in the container
      # This allows code changes without rebuilding the image during development
      - ./app:/app
    depends_on:
      - redis # Ensures redis starts before the api service
    networks:
      - ml_app_network # Attach this service to the custom network

  # Service 2: The Redis Cache
  redis:
    image: "redis:6.2-alpine" # Use a standard Redis image from Docker Hub
    ports:
      - "6379:6379" # Expose Redis port for potential direct inspection (optional)
    volumes:
      - redis_data:/data # Mount a named volume for data persistence
    networks:
      - ml_app_network # Attach this service to the custom network

# Define named volumes
volumes:
  redis_data: # Persists Redis data even if the container is removed

# Define custom network
networks:
  ml_app_network:
    driver: bridge # Use the default bridge driver

Let's break down this docker-compose.yml file:

• version: '3.8': Specifies the version of the Compose file format.
• services:: Defines the containers that make up our application.
  • api:: Defines our Flask application service.
    • build: .: Tells Compose to build an image using the Dockerfile in the current directory.
    • ports: - "5001:5000": Maps port 5001 on the host machine to port 5000 inside the api container (where Flask runs). We use 5001 on the host to avoid potential conflicts with other services running on 5000.
    • environment:: Sets environment variables inside the api container. Importantly, REDIS_HOST=redis tells our Flask app to connect to the service named redis. Compose handles the DNS resolution.
    • volumes: - ./app:/app: Mounts the local app directory into the container's /app directory. Changes made locally to the Python code in app/ will be reflected immediately in the running container, which is very useful for development.
    • depends_on: - redis: Specifies that the api service depends on the redis service. Compose will start redis first and wait for it to be running before starting api. Note: depends_on only waits for the container to start, not necessarily for the application inside it to be ready. More robust checks often involve health checks or wait scripts.
    • networks: - ml_app_network: Connects the service to our defined network.
  • redis:: Defines our Redis cache service.
    • image: "redis:6.2-alpine": Tells Compose to pull the specified Redis image from Docker Hub. The alpine tag indicates a smaller image variant.
    • ports: - "6379:6379": Optionally map the standard Redis port for direct access from the host if needed for debugging.
    • volumes: - redis_data:/data: Mounts a named volume redis_data to the /data directory inside the Redis container, which is where Redis stores its data. This ensures that cached data persists even if the redis container is stopped and restarted.
    • networks: - ml_app_network: Connects the service to our defined network.
• volumes:: Declares the named volume redis_data. Docker manages this volume.
• networks:: Declares a custom bridge network ml_app_network. Using custom networks is recommended practice, providing better isolation and enabling DNS resolution between service names (api, redis).

Running the Application

Navigate to your compose_ml_app directory in the terminal.

1. Build and Start Services: Run the following command:

   docker compose up --build
   

   • docker compose up: This command reads the docker-compose.yml file, creates the network and volume (if they don't exist), builds the image for the api service (because of build: .), pulls the redis image, and starts both containers.
   • --build: Forces Docker Compose to build the image for services defined with the build instruction, even if an image with the same name already exists. Useful when you've changed the Dockerfile or application code dependencies.

   You will see logs from both the api and redis containers interleaved in your terminal. Look for lines indicating that Redis is ready and the Flask app is running (e.g., * Running on http://0.0.0.0:5000/).

2. Test the API: Open another terminal window and use curl (or any HTTP client) to send a POST request to the /predict endpoint of your API service, which is accessible on port 5001 of your host machine:

   # First request (cache miss)
   curl -X POST -H "Content-Type: application/json" \
        -d '{"input": "some_feature_vector_1"}' \
        http://localhost:5001/predict
   
   # Expected output (might vary slightly):
   # {"prediction":"prediction_for_some_feature_vector_1","source":"computation"}
   

   Check the logs from docker compose up. You should see messages from the Flask app indicating a "Cache miss" and then storing the result.

   Now, send the same request again:

   # Second request (cache hit)
   curl -X POST -H "Content-Type: application/json" \
        -d '{"input": "some_feature_vector_1"}' \
        http://localhost:5001/predict
   
   # Expected output:
   # {"prediction":"prediction_for_some_feature_vector_1","source":"cache"}
   

   This time, the response indicates the result came from the "cache", and the logs should show a "Cache hit" message. This confirms that the api service successfully communicated with the redis service over the Docker network defined by Compose.

   You can also test the health check endpoint:

   curl http://localhost:5001/health
   # Expected output:
   # {"cache_connected":true,"status":"ok"}
   

3. Stopping the Application: Go back to the terminal where docker compose up is running and press Ctrl+C. This will stop the containers.

   To ensure the containers and the network are removed (the named volume redis_data will persist by default), run:

   docker compose down
   

   If you also want to remove the named volume, use:

   docker compose down --volumes
   

Visualizing the Setup

We can represent our Compose setup with a simple diagram:

This diagram shows the client (curl) interacting with the api service via the mapped host port. The api service communicates with the redis service using the service name over the internal Docker network (ml_app_network). The redis service uses a named volume (redis_data) for persistence.

This practical exercise demonstrates how Docker Compose simplifies managing interconnected services. By defining our application stack in docker-compose.yml, we automated the creation of networks, volumes, builds, and container startup order, making our development workflow much more efficient and reproducible compared to managing individual docker run commands. You can adapt this pattern for more complex ML applications involving databases, message queues, monitoring tools, or other components.