ApX logoApX logo
ModelsCalculatorHardwareCoursesPricing
ApX logoApX logo
ModelsCalculatorHardwareCoursesPricing

All Courses

Python for LLM Workflows: Tooling and Best Practices
Chapter 1: Introduction to LLM Workflows with Python
What are LLM Workflows?
Common Components of an LLM Application
Why Python for LLM Development?
Overview of the Python LLM Ecosystem
Course Structure and Goals
Quiz for Chapter 1
Chapter 2: Setting Up Your Python Environment for LLM Development
Choosing Your Python Version
Virtual Environments (venv, conda)
Managing Dependencies with pip and requirements.txt
Essential Libraries Installation
Setting Up API Keys Securely
Development Environment Configuration Practice
Quiz for Chapter 2
Chapter 3: Interacting with LLM APIs using Python
Understanding LLM APIs
Making API Requests with the requests Library
Handling API Responses and Errors
Using Official Python Client Libraries
Authentication Methods
Rate Limiting and Cost Management Considerations
Practice: Querying an LLM API
Quiz for Chapter 3
Chapter 4: Core LLM Workflow Libraries: LangChain Fundamentals
Introduction to LangChain
Core Concepts: Models, Prompts, Output Parsers
Working with Different LLM Providers in LangChain
Creating and Using Prompt Templates
Parsing LLM Output Structures
Hands-on Practical: Building a Simple LangChain Application
Quiz for Chapter 4
Chapter 5: Advanced LangChain: Chains and Agents
Understanding Chains for Sequential Operations
Building Custom Chains
Introduction to Agents: LLMs as Reasoning Engines
Available Tools for Agents
Creating a Basic Agent
Debugging Chains and Agents
Practice: Implementing a Multi-Step Chain
Quiz for Chapter 5
Chapter 6: Data Handling for LLMs: LlamaIndex Basics
Introduction to LlamaIndex
Loading Data (Documents, Web Pages)
Indexing Data for Efficient Retrieval
Understanding Nodes and Indexes
Querying Your Indexed Data
Hands-on Practical: Indexing and Querying Documents
Quiz for Chapter 6
Chapter 7: Building Retrieval-Augmented Generation (RAG) Systems
Retrieval-Augmented Generation Concepts
Integrating LlamaIndex/LangChain for RAG
Vector Stores and Embeddings Overview
Setting Up a Basic Vector Store
Constructing a RAG Pipeline
Evaluating RAG Performance Metrics
Practice: Creating a Simple RAG Application
Quiz for Chapter 7
Chapter 8: Prompt Engineering Techniques in Python
Principles of Effective Prompting
Few-Shot Prompting Techniques
Structuring Prompts for Complex Tasks
Using Python for Dynamic Prompt Generation
Techniques for Reducing Hallucinations
Iterative Prompt Refinement
Practice: Developing and Testing Prompts
Quiz for Chapter 8
Chapter 9: Testing and Evaluating LLM Applications
Challenges in Testing LLM Systems
Unit Testing Components
Integration Testing Workflows
Evaluation Strategies: Metrics and Human Feedback
Using Frameworks for Evaluation
Logging and Monitoring LLM Interactions
Practice: Setting Up Basic Tests for an LLM Chain
Quiz for Chapter 9
Chapter 10: Deployment and Operational Best Practices
Packaging Your Python LLM Application
Containerization with Docker
Choosing a Deployment Strategy
API Endpoint Creation (FastAPI, Flask)
Monitoring Deployed Applications
Version Control and CI/CD for LLM Projects
Operational Considerations and Maintenance
Quiz for Chapter 10
Domain Specific Intelligent Knowledge Assistant Construction

Handling API Responses and Errors

When interacting with LLM APIs, proper handling of responses and potential errors is essential. Applications frequently encounter unexpected behaviors or errors, and simply assuming the request worked and the data is ready often leads to significant bugs. Therefore, applications require robust mechanisms to gracefully manage both successful API responses and various error conditions.

Checking the Response Status

The first thing to check is the HTTP status code returned by the API server. Web standards define ranges for these codes:

  • 2xx (e.g., 200 OK): Indicates success. The request was received, understood, and accepted.
  • 4xx (e.g., 400 Bad Request, 401 Unauthorized, 404 Not Found, 429 Too Many Requests): Client errors. Something was wrong with the request sent from your application (malformed data, missing authentication, requested resource doesn't exist, rate limit exceeded).
  • 5xx (e.g., 500 Internal Server Error, 503 Service Unavailable): Server errors. Something went wrong on the API provider's end.

The requests library makes checking for success straightforward. The Response object has a status_code attribute and a boolean ok attribute which is True if the status code is less than 400 (i.e., success or redirect), and False otherwise.

import requests
import os
import json

# Assume API_ENDPOINT and API_KEY are set appropriately
# API_ENDPOINT = "YOUR_LLM_API_ENDPOINT"
# API_KEY = os.getenv("LLM_API_KEY") # Example of getting key securely

headers = {
    "Authorization": f"Bearer {API_KEY}",
    "Content-Type": "application/json"
}

data = {
    "prompt": "Explain the difference between HTTP status codes 200 and 404.",
    "max_tokens": 50
}

try:
    # response = requests.post(API_ENDPOINT, headers=headers, json=data, timeout=10) # Example call
    # For demonstration, let's simulate a response object
    class MockResponse:
        def __init__(self, status_code, text_content, is_json=True):
            self.status_code = status_code
            self.text = text_content
            self._is_json = is_json

        @property
        def ok(self):
            return self.status_code < 400

        def json(self):
            if not self._is_json:
                raise requests.exceptions.JSONDecodeError("Expecting value", "dummy", 0)
            try:
                return json.loads(self.text)
            except json.JSONDecodeError as e:
                 raise requests.exceptions.JSONDecodeError(e.msg, e.doc, e.pos)

    # Simulate a successful response
    response = MockResponse(200, '{"completion": "200 OK means success, 404 Not Found means the resource doesn\'t exist."}', is_json=True)

    # Always check the status first
    if response.ok:
        print(f"Request successful (Status Code: {response.status_code})")
        # Proceed to parse the body
    else:
        print(f"Request failed (Status Code: {response.status_code})")
        print(f"Response Body: {response.text}")
        # Handle specific error codes (4xx, 5xx) here

except requests.exceptions.RequestException as e:
    print(f"An error occurred during the request: {e}")

Always check response.ok or response.status_code before attempting to process the response body. Trying to parse JSON from an error response (like a 404 page which might be HTML) will likely raise an exception.

Parsing the Response Body

If the status code indicates success (usually 200 OK), the LLM's output is typically contained within the response body, often formatted as JSON. The requests library provides the response.json() method to conveniently parse this.

# Continuing from the successful response simulation above...

if response.ok:
    try:
        # Parse the JSON response
        response_data = response.json()
        print("Successfully parsed JSON response:")
        # print(json.dumps(response_data, indent=2)) # Pretty print

        # Extract the relevant information (this structure varies by API)
        if "completion" in response_data:
             llm_output = response_data["completion"]
             print(f"LLM Output: {llm_output}")
        elif "choices" in response_data and len(response_data["choices"]) > 0:
             # Handling structure like OpenAI's chat completions
             first_choice = response_data["choices"][0]
             if "message" in first_choice and "content" in first_choice["message"]:
                 llm_output = first_choice["message"]["content"]
                 print(f"LLM Output: {llm_output}")
             else:
                 print("Could not find expected message content in the first choice.")
        else:
            print("Response JSON does not contain expected 'completion' or 'choices' field.")
            print("Full Response:", response_data)

    except requests.exceptions.JSONDecodeError:
        # Handle cases where the response status was OK, but the body wasn't valid JSON
        print(f"Failed to decode JSON, even though status was {response.status_code}.")
        print(f"Raw Response Text: {response.text}")
    except KeyError as e:
        # Handle cases where the JSON is valid, but missing expected keys
        print(f"Missing expected key {e} in the JSON response.")
        print("Full Response:", response_data)

else:
     # Handle non-OK status codes as before
     print(f"Request failed (Status Code: {response.status_code})")
     print(f"Response Body: {response.text}")

Notice the nested try...except block. It's possible to get a 200 OK status but receive a response body that isn't valid JSON, or valid JSON that doesn't contain the keys you expect (e.g., completion, choices, message). Your code needs to anticipate these possibilities. The exact structure of the JSON response varies significantly between different LLM providers and API endpoints, so consult the specific API documentation you are using.

Handling API-Specific Errors

Sometimes, an API might return a 200 OK status code but still indicate an error within the JSON payload itself. For example, the API might accept your request structurally but report an issue like an invalid parameter value, a prompt being flagged by a content filter, or exceeding a specific quota.

{
  "error": {
    "code": "invalid_prompt",
    "message": "The provided prompt was rejected by the content filter.",
    "type": "validation_error"
  }
}

It's important to check for such error structures within the JSON after successfully parsing it, even if the HTTP status was 200 OK.

# Assuming response.ok was True and response_data = response.json() was successful

if "error" in response_data:
    error_info = response_data["error"]
    print(f"API returned an error within the JSON payload:")
    print(f"  Code: {error_info.get('code', 'N/A')}")
    print(f"  Message: {error_info.get('message', 'No message provided.')}")
    # Implement specific handling based on the error code or type
else:
    # Process the successful data as shown previously
    print("No API-specific error found in payload. Processing data...")
    # ... extract completion, choices, etc. ...

Common Errors and Basic Handling Strategies

Let's summarize common errors and suggest basic handling:

  1. Network/Connection Errors (requests.exceptions.RequestException): These occur before you even get a response (timeout, DNS failure, connection refused).

    • Handling: Use a try...except requests.exceptions.RequestException block around your requests.post or requests.get call. Log the error. You might retry the request after a short delay, potentially with exponential backoff (waiting longer after each failed attempt).
  2. Authentication Errors (401 Unauthorized, 403 Forbidden): Your API key might be invalid, expired, or lack permissions for the requested operation.

    • Handling: Stop execution for this request. Log the error clearly. Verify your API key and permissions. Do not retry automatically with the same credentials. Alerting a developer might be necessary.
  3. Rate Limit Errors (429 Too Many Requests): You've sent too many requests in a given time window.

    • Handling: Stop sending requests for a period. The API response often includes a Retry-After header indicating how long to wait (in seconds). If not present, implement exponential backoff (e.g., wait 1s, then 2s, 4s, 8s, etc.) before retrying.
  4. Client Errors (Other 4xx, e.g., 400 Bad Request): Your request might be malformed (invalid JSON, missing required parameters).

    • Handling: Stop execution for this request. Log the error and the response body, which often contains details about what was wrong. Fix the request structure in your code. Do not retry automatically.
  5. Server Errors (5xx, e.g., 500 Internal Server Error, 503 Service Unavailable): The API provider is having temporary issues.

    • Handling: These errors are often transient. Retry the request after a delay, using exponential backoff. If the error persists after several retries, log it and potentially alert operations staff.
  6. JSON Decode Errors (requests.exceptions.JSONDecodeError): The response body couldn't be parsed as JSON.

    • Handling: Log the error and the raw response.text. This might indicate an unexpected response format (like HTML from an error page) or a genuine server issue producing malformed JSON.
  7. API-Specific Errors (in JSON Payload): Errors reported within a 200 OK response.

    • Handling: Parse the error details from the JSON. Log the specific error code and message. Handle based on the error type (e.g., modify the prompt if it violated content policy, stop if a quota is exceeded).

A typical flow for handling API responses and potential errors.

Effectively handling API responses and errors is essential for building reliable LLM applications. By checking status codes, carefully parsing expected data formats, and anticipating various failure modes, you can create more resilient Python code that interacts predictably with LLM services. Remember to consult the specific API documentation for details on response structures and error codes.

  • HTTP response status codes, MDN Web Docs Contributors, 2024 - An authoritative explanation of HTTP status codes, detailing their purpose in web communication.
  • Requests: HTTP for Humans™ Documentation, The Requests Community, 2024 - The official documentation for the Python requests library, covering its API, response objects, and exception handling.
  • Error Retries and Exponential Backoff in AWS, Amazon Web Services, 2023 - Offers instructions for retry mechanisms using exponential backoff in API interactions, supporting reliable client applications.
  • json - JSON encoder and decoder, Python Software Foundation, 2024 - Documentation for Python's built-in json module, which is for parsing and serializing JSON data.

© 2026 ApX Machine Learning