Hands-on Practical: Creating a Synthetic Dataset for Task-Specific Fine-Tuning

This hands-on exercise focuses on creating a small, targeted synthetic dataset for a specific fine-tuning task. It involves generating instruction-response pairs designed to shape an LLM's behavior, specifically to act as an empathetic customer service agent. The exercise covers defining the task, crafting generation prompts, producing data, and structuring it into a usable format for fine-tuning pipelines.

Defining Your Fine-Tuning Goal

Before generating any data, it's important to clearly define what you want your fine-tuned LLM to do. A precise goal helps in crafting effective prompts and evaluating the quality of your synthetic data.

For this practical, our goal is to create a dataset to fine-tune an LLM to become an Empathetic Customer Service Agent.

• Task: Respond to customer queries with empathy and helpfulness.
• Input to the LLM (after fine-tuning): A customer's statement or question.
• Desired LLM Output: A response that acknowledges the customer's feelings, shows understanding, and offers concrete assistance or next steps.
• Targeted Behaviors: Patience, active listening (simulated), problem-solving, and a consistently supportive tone.

Having this clear objective will guide every step of our synthetic data generation process.

Choosing Your Generation Method

As discussed earlier in this chapter, and in Chapter 2, there are several ways to generate synthetic text. These include rule-based systems, back-translation, paraphrasing, and using other LLMs. For tasks requiring specific language details, specific personas, or complex instruction following, using a powerful LLM as the generator (often called "LLM-as-a-judge" or simply LLM-based generation) is a highly effective approach.

We will use an LLM to generate our synthetic dataset. This method offers great flexibility and can produce high-quality, diverse examples when guided by well-designed prompts.

Crafting Seed Examples (Optional but Recommended)

While a sufficiently detailed prompt can guide an LLM, providing a few high-quality examples (seed examples) within your generation prompt can significantly improve the output. These examples act as a template, showing the LLM the desired structure, style, tone, and complexity of the instruction-response pairs you want.

Here are a couple of seed examples for our empathetic customer service agent task:

1. Customer Query (Instruction Part): "My order #ORD12345 was supposed to arrive three days ago, but the tracking hasn't updated. I'm really worried." Empathetic Response (Output Part): "I completely understand your concern about order #ORD12345, especially when tracking hasn't updated. That must be frustrating. Let me look into this for you right away. I'll check the latest shipping information and see what we can do to resolve this."

2. Customer Query (Instruction Part): "I received the product, but it's not working as advertised. This is so disappointing!" Empathetic Response (Output Part): "Oh, I'm truly sorry to hear that the product isn't working as expected. I can see how disappointing that would be after anticipating its arrival. We definitely want to make this right for you. Could you please tell me a bit more about what's happening with it?"

These seeds demonstrate empathy, a proactive approach, and a request for more information to help solve the problem.

Engineering the Generation Prompt

The prompt you provide to the generator LLM is the most important element in this process. It needs to clearly instruct the LLM on what to create. For our task, we want the LLM to generate new customer queries and corresponding empathetic responses.

Here's an example of a meta-prompt structure you could use. This prompt asks the LLM to generate data in a specific JSON structure, which can simplify parsing later.

You are an assistant helping to create a dataset for fine-tuning a large language model to be an empathetic customer service agent. Your task is to generate diverse examples of customer queries and ideal empathetic responses.

Please generate a new, unique customer query and an appropriate empathetic, helpful response. The query should sound like it's from a real customer, and the response should demonstrate understanding, acknowledge feelings, and offer clear next steps or assistance.

Here are some examples of the style and content we are looking for:

Example 1:
Customer Query: "My order #ORD12345 was supposed to arrive three days ago, but the tracking hasn't updated. I'm really worried."
Empathetic Response: "I completely understand your concern about order #ORD12345, especially when tracking hasn't updated. That must be frustrating. Let me look into this for you right away. I'll check the latest shipping information and see what we can do to resolve this."

Example 2:
Customer Query: "I received the product, but it's not working as advertised. This is so disappointing!"
Empathetic Response: "Oh, I'm truly sorry to hear that the product isn't working as expected. I can see how disappointing that would be after anticipating its arrival. We definitely want to make this right for you. Could you please tell me a bit more about what's happening with it?"

Now, please generate a new example. Ensure the customer query is different from the examples provided. The response should maintain an empathetic tone.

Provide your output as a JSON object with two keys: "customer_query" and "empathetic_response".
For instance:
{
  "customer_query": "A new customer query here...",
  "empathetic_response": "A new empathetic response here..."
}

Main elements in this prompt:

• Role Setting: Defines the LLM's purpose.
• Task Description: Clearly states what needs to be generated.
• Desired Qualities: Emphasizes empathy, helpfulness, diversity.
• Few-Shot Examples: Provides concrete illustrations (our seed examples).
• Output Format Instruction: Asks for JSON, which makes the output easy to use.
• Instruction for Novelty: Explicitly asks for new and unique examples.

You would then send this prompt to your chosen LLM (e.g., GPT-4, Claude 3, Gemini Pro, or an open-source model).

Generating the Synthetic Data

To generate multiple data points, you would typically write a script that repeatedly calls an LLM API with your engineered prompt. For each call, you might slightly vary the prompt (e.g., by asking for a query about a specific topic like "billing issue" or "product feature") or rely on the LLM's inherent creativity and temperature settings to produce diverse outputs.

Here's a Python snippet illustrating how you might make such calls and collect responses. This example uses a placeholder call_llm_api function. In a real application, you'd use a library like openai, anthropic, or google-generativeai.

import json
import time # To avoid hitting rate limits

# This is a placeholder for your actual LLM API call function.
# It would take the prompt and return the LLM's text response.
def call_llm_api(prompt_text):
    """
    Simulates a call to an LLM API.
    In a real scenario, this function would use an API client (OpenAI, Anthropic, etc.)
    to send the prompt_text to an LLM and return its response.
    For this example, we simulate a JSON string output as requested by our prompt.
    """
    print("Simulating LLM call...")
    # Based on the prompt, we expect a JSON string.
    # Let's simulate a plausible output for a new query.
    simulated_json_output = {
        "customer_query": "I'm trying to use a discount code you sent me, but it says it's invalid. What's wrong?",
        "empathetic_response": "I'm sorry to hear you're having trouble with the discount code! That's definitely frustrating when you're looking forward to a good deal. I can help with that. Could you please share the discount code with me so I can check its status and details?"
    }
    # In a real API call, you might get a more complex response object.
    # Here, we assume the core content is easily extractable or already structured.
    return json.dumps(simulated_json_output)

generation_prompt_template = """
You are an assistant helping to create a dataset for fine-tuning a large language model to be an empathetic customer service agent. Your task is to generate diverse examples of customer queries and ideal empathetic responses.

Please generate a new, unique customer query and an appropriate empathetic, helpful response. The query should sound like it's from a real customer, and the response should demonstrate understanding, acknowledge feelings, and offer clear next steps or assistance.

Here are some examples of the style and content we are looking for:

Example 1:
Customer Query: "My order #ORD12345 was supposed to arrive three days ago, but the tracking hasn't updated. I'm really worried."
Empathetic Response: "I completely understand your concern about order #ORD12345, especially when tracking hasn't updated. That must be frustrating. Let me look into this for you right away. I'll check the latest shipping information and see what we can do to resolve this."

Example 2:
Customer Query: "I received the product, but it's not working as advertised. This is so disappointing!"
Empathetic Response: "Oh, I'm truly sorry to hear that the product isn't working as expected. I can see how disappointing that would be after anticipating its arrival. We definitely want to make this right for you. Could you please tell me a bit more about what's happening with it?"

Now, please generate a new example. Ensure the customer query is different from the examples provided. The response should maintain an empathetic tone.

Provide your output as a JSON object with two keys: "customer_query" and "empathetic_response".
"""

synthetic_data_points = []
number_of_examples_to_generate = 5 # For a real dataset, this would be much larger

for i in range(number_of_examples_to_generate):
    print(f"Generating example {i+1}/{number_of_examples_to_generate}...")
    # You could add slight variations to the prompt here if desired,
    # for instance, by asking for queries related to different themes.
    # For simplicity, we use the same prompt.

    raw_llm_output = call_llm_api(generation_prompt_template)

    if raw_llm_output:
        try:
            data_point = json.loads(raw_llm_output)
            if "customer_query" in data_point and "empathetic_response" in data_point:
                synthetic_data_points.append(data_point)
                print(f"Successfully generated: {data_point['customer_query'][:50]}...")
            else:
                print("Error: LLM output did not contain expected keys.")
        except json.JSONDecodeError:
            print(f"Error: Could not decode LLM output as JSON: {raw_llm_output}")
    else:
        print("Error: LLM API call failed or returned no output.")

    time.sleep(1) # Respect API rate limits if using a real API

print(f"\nGenerated {len(synthetic_data_points)} data points.")
# At this point, synthetic_data_points contains a list of dictionaries.

This script would call the LLM API multiple times. Each raw_llm_output is expected to be a JSON string like {"customer_query": "...", "empathetic_response": "..."}. The script then parses this string into a Python dictionary and appends it to our list.

Structuring Data for Fine-Tuning

Most LLM fine-tuning frameworks expect data in a specific format, commonly JSON Lines (JSONL). In a JSONL file, each line is a valid JSON object. For instruction fine-tuning, a popular format is the one used by Alpaca, which includes instruction, input, and output fields.

Let's transform our generated customer_query and empathetic_response pairs into this Alpaca-style format.

• The instruction will be a general directive to the model.
• The input will be the specific customer query.
• The output will be the desired empathetic response.

Here’s how you can convert the synthetic_data_points list into a JSONL file:

# (Continuing from the previous Python script)

# Define the general instruction for the fine-tuning task
general_instruction = "You are an empathetic customer service agent. Provide a helpful and understanding response to the customer's query."

fine_tuning_dataset = []
for item in synthetic_data_points:
    # Ensure we have the expected keys from our generation step
    if "customer_query" in item and "empathetic_response" in item:
        fine_tuning_entry = {
            "instruction": general_instruction,
            "input": item["customer_query"],
            "output": item["empathetic_response"]
        }
        fine_tuning_dataset.append(fine_tuning_entry)

# Save to a JSONL file
output_filename = "empathetic_customer_service_dataset.jsonl"
with open(output_filename, 'w') as f:
    for entry in fine_tuning_dataset:
        json.dump(entry, f)
        f.write('\n')

print(f"\nFormatted dataset saved to {output_filename}")

An example line in your empathetic_customer_service_dataset.jsonl file would look like this:

{"instruction": "You are an empathetic customer service agent. Provide a helpful and understanding response to the customer's query.", "input": "I'm trying to use a discount code you sent me, but it says it's invalid. What's wrong?", "output": "I'm sorry to hear you're having trouble with the discount code! That's definitely frustrating when you're looking forward to a good deal. I can help with that. Could you please share the discount code with me so I can check its status and details?"}

This JSONL file is now ready to be used with many popular fine-tuning libraries and platforms.

Workflow Diagram

The process we followed can be summarized in the following diagram:

This diagram illustrates the workflow from initial seed examples through prompt engineering, LLM-based generation, data structuring, and finally, the creation of a ready-to-use fine-tuning dataset.

Initial Review and Next Steps

You've now created a small synthetic dataset! Before using it for actual fine-tuning, it's good practice to:

1. Manually Inspect Samples: Read through a portion of your generated data. Does it meet your quality criteria? Is it diverse enough? Are there any repetitive patterns or undesirable outputs?
2. Check for Relevance: Does each instruction-response pair align with the intended task (empathetic customer service)?
3. Consider Data Augmentation: If your initial set is too small or lacks diversity, you might refine your generation prompt, use more varied seed examples, or employ techniques from Chapter 2 to augment your data further.

This hands-on exercise demonstrates a fundamental pipeline for creating synthetic data for fine-tuning. As you scale up, you'll iterate on prompt design, explore different generation strategies, and implement more rigorous quality checks, which we'll cover in later chapters, particularly Chapter 6 on evaluation. For now, you have a solid foundation for generating targeted datasets to customize LLMs for specific needs.