Practice: Querying an LLM API

Hands-on steps are provided to send a query to an LLM API using Python and interpret the result. Proficiency in LLM API fundamentals, authentication, and response handling is recommended. Before proceeding, ensure your development environment is set up and your API key is secured.

Prerequisites

Before proceeding, ensure you have:

1. An API Key: Obtain an API key from an LLM provider (like OpenAI, Anthropic, Cohere, or others).
2. API Key Secured: Set your API key as an environment variable (e.g., OPENAI_API_KEY). This is important for security and avoids hardcoding keys in your script. Refer back to Chapter 2 if you need a refresher on setting environment variables.
3. Libraries Installed: Install the necessary Python library. We'll show examples using both the standard requests library and a vendor-specific library (using OpenAI's openai library as an illustration).

   pip install requests python-dotenv openai
   

4. .env File: Create a file named .env in your project directory and add your API key like this:

   OPENAI_API_KEY='your_api_key_here'
   

Querying using the requests Library

The requests library gives you direct control over the HTTP request. This is useful for understanding the underlying communication but can be more verbose than using a dedicated client library.

import requests
import os
import json
from dotenv import load_dotenv

# Load environment variables from .env file
load_dotenv()

# Retrieve the API key from environment variables
api_key = os.getenv("OPENAI_API_KEY")
if not api_key:
    raise ValueError("API key not found. Set the OPENAI_API_KEY environment variable.")

# Define the API endpoint (Using OpenAI's Chat Completions endpoint as an example)
api_url = "https://api.openai.com/v1/chat/completions"

# Prepare the headers with Authorization
headers = {
    "Content-Type": "application/json",
    "Authorization": f"Bearer {api_key}"
}

# Prepare the data payload
data = {
    "model": "gpt-3.5-turbo", # Specify the model you want to use
    "messages": [
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Explain the difference between an API and an SDK in simple terms."}
    ],
    "temperature": 0.7, # Controls randomness (0=deterministic, 1=more random)
    "max_tokens": 100 # Limit the length of the response
}

try:
    # Send the POST request
    response = requests.post(api_url, headers=headers, json=data, timeout=30) # Added timeout

    # Check if the request was successful
    response.raise_for_status() # Raises an HTTPError for bad responses (4XX or 5XX)

    # Parse the JSON response
    result = response.json()

    # Print the relevant part of the response
    print("Response using requests:")
    if result.get("choices"):
        print(result["choices"][0]["message"]["content"])
    else:
        print("No choices found in the response.")
    # Optional: Print usage information if available
    if result.get("usage"):
        print("\nUsage Info:", result["usage"])

except requests.exceptions.RequestException as e:
    print(f"An error occurred during the API request: {e}")
except Exception as e:
    print(f"An unexpected error occurred: {e}")

This script first loads the API key securely. It then defines the target API endpoint and constructs the necessary headers, including the authorization token. The data dictionary contains the prompt ("Explain the difference...") along with parameters like the model name, temperature, and maximum tokens. Finally, it sends the request, checks for errors, parses the JSON response, and prints the generated text.

Querying using an Official Client Library (OpenAI Example)

Official client libraries (SDKs) abstract away much of the HTTP request boilerplate, offering a more Pythonic interface.

import os
from openai import OpenAI, OpenAIError
from dotenv import load_dotenv

# Load environment variables from .env file
load_dotenv()

# The OpenAI library automatically looks for the OPENAI_API_KEY environment variable.
# You can also initialize it explicitly: client = OpenAI(api_key=os.getenv("OPENAI_API_KEY"))
try:
    client = OpenAI() # Initializes client, automatically uses OPENAI_API_KEY env var

    # Make the API call
    completion = client.chat.completions.create(
        model="gpt-3.5-turbo",
        messages=[
            {"role": "system", "content": "You are a helpful assistant."},
            {"role": "user", "content": "Explain the difference between an API and an SDK in simple terms."}
        ],
        temperature=0.7,
        max_tokens=100
    )

    # Print the relevant part of the response
    print("\nResponse using OpenAI client library:")
    if completion.choices:
        print(completion.choices[0].message.content)
    else:
        print("No choices found in the response.")

    # Optional: Print usage information
    if completion.usage:
        print("\nUsage Info:", completion.usage)

except OpenAIError as e:
    print(f"An OpenAI API error occurred: {e}")
except Exception as e:
    print(f"An unexpected error occurred: {e}")

Notice how much cleaner this code is. The openai library handles setting the headers and structuring the request internally. You simply instantiate the client and call the appropriate method (chat.completions.create). Error handling specific to the OpenAI API is also often built into the library.

Understanding the Interaction Flow

The process involves your Python script sending structured data (your prompt and parameters) to the LLM provider's API endpoint, which then processes the request and returns the generated text and metadata.

A simple diagram illustrating the request-response cycle between your Python script and the LLM API service.

Experiment and Observe

Now it's your turn to experiment. Modify the code examples:

• Change the prompt: Ask different questions or give different instructions.
• Adjust temperature: Try values like 0.1 (more focused) and 1.0 (more creative/random) and see how the output changes.
• Modify max_tokens: Set a smaller value (e.g., 20) or a larger one (within the model's limits) to control response length.
• Use a different model: If your API key grants access, try specifying another model (e.g., "gpt-4" if using OpenAI and you have access).
• Introduce Errors: Temporarily use an invalid API key or malform the request data to see how the error handling (try...except) blocks work.

Running these experiments will solidify your understanding of how to control LLM behavior through API parameters. Remember the points about rate limiting and cost management discussed earlier; frequent or complex queries will consume your quota and potentially incur costs.

By completing this practice session, you have successfully used Python to interact directly with an LLM API. You can now send prompts, configure generation parameters, and receive responses programmatically. This forms a fundamental skill for building more sophisticated LLM workflows, which we will begin constructing in the next chapters using libraries like LangChain.