Defining Custom Evaluation Metrics

While standard machine learning metrics like accuracy, precision, and recall provide a baseline, they often fall short when assessing the multifaceted performance of applications built with Large Language Models (LLMs). LLM outputs possess qualities like coherence, relevance, helpfulness, safety, and adherence to specific formats or tones, which are difficult to capture with simple quantitative measures. Evaluating a production LangChain application effectively requires defining custom metrics tailored to its specific function and desired behavior.

Moving beyond simple correctness checks allows you to measure what truly matters for your application's success. For instance, is your RAG system retrieving relevant context and grounding its answers faithfully? Is your customer support agent maintaining a helpful and safe tone? Is your summarization tool capturing the essence without distortion? Answering these requires bespoke evaluation strategies.

Why Standard Metrics Are Often Insufficient

Traditional metrics typically rely on exact matches or predefined classifications. LLM outputs, however, are often generative and nuanced:

1. Subjectivity: Qualities like "helpfulness," "creativity," or "tone" are inherently subjective and context-dependent.
2. Semantic Equivalence: An LLM might generate a perfectly valid answer that uses different phrasing than the "ground truth" reference, failing an exact match test despite being correct.
3. Safety and Bias: Standard metrics rarely account for harmful content generation, bias, or ethical alignment.
4. Format Adherence: Applications might require outputs in specific formats (e.g., JSON, specific XML structure) which isn't measured by semantic correctness alone.
5. Contextual Relevance (RAG): Assessing Retrieval-Augmented Generation involves evaluating both the retrieved documents' relevance and the final answer's faithfulness to that context.

Therefore, developing custom evaluation metrics becomes a significant step in building reliable LLM applications.

Categories of Custom Evaluation Metrics

Custom metrics can be broadly categorized based on how they assess the output:

1. Programmatic & Rule-Based Metrics: These involve writing code to check specific, objective criteria.

   • Format Validation: Does the output adhere to the required JSON schema, XML structure, or list format? Use standard parsing libraries (e.g., json, xml.etree.ElementTree) within your evaluation function.
   • Length Constraints: Is the summary within the specified word/token count?
   • Keyword Presence/Absence: Does the output contain required keywords or avoid forbidden ones (e.g., checking for PII markers, ensuring specific brand terms are included)?
   • API Call Validation (Agents): Did the agent call the correct tool API with valid arguments? Check the agent trajectory for tool names and input parameters.

2. Semantic Similarity Metrics: These use embeddings to measure the semantic closeness between the generated output and a reference answer or the input query/context.

   • Embedding Distance: Generate embeddings (using models like Sentence-BERT or OpenAI embeddings) for the generated output and the reference answer. Calculate cosine similarity or Euclidean distance. A high similarity suggests semantic equivalence. $$\text{Cosine Similarity}(A, B) = \frac{A \cdot B}{\|A\| \|B\|}$$
   • Context Relevance (RAG): Calculate the semantic similarity between the input query and the retrieved document chunks.
   • Faithfulness (RAG): Measure the semantic similarity between the generated answer and the retrieved context it's supposed to be based on. Low similarity might indicate hallucination.

3. Model-Based Evaluation (LLM-as-Judge): This powerful technique uses another LLM (often a capable one like GPT-4) to evaluate the output based on specific criteria defined in a prompt.

   • Prompting for Scores: Design a prompt instructing the judge LLM to rate the output on a numerical scale (e.g., 1-5) for criteria like helpfulness, coherence, safety, or factual accuracy. Include the input, output, and potentially reference answer/context in the judge's prompt.
   • Prompting for Explanations: Ask the judge LLM not just for a score but also for a textual explanation of its reasoning. This provides qualitative insights.
   • Aspect-Specific Evaluation: Use separate judge prompts to evaluate different aspects independently (e.g., one for tone, one for factual accuracy).

   # Simplified example of an LLM-as-judge prompt structure
   EVALUATION_PROMPT = """
   You are an impartial evaluator. Assess the quality of the submitted 'RESPONSE'
   based on the provided 'CONTEXT' and 'QUERY'.
   
   QUERY: {query}
   CONTEXT: {context}
   RESPONSE: {response}
   
   Evaluate the RESPONSE based on the following criteria on a scale of 1 to 5 (1=Poor, 5=Excellent):
   1. Faithfulness: Does the RESPONSE accurately reflect the information in the CONTEXT without adding unsupported claims?
   2. Relevance: Is the RESPONSE directly relevant to the QUERY?
   
   Provide your scores in JSON format: {"faithfulness": score, "relevance": score}
   """
   

4. Human-in-the-Loop (HITL): Collecting feedback directly from humans remains the gold standard for subjective qualities.

   • Direct Feedback: Integrate mechanisms (e.g., thumbs up/down buttons, rating scales, comment boxes) into your application for users to provide feedback.
   • Annotation Platforms: Use platforms (including LangSmith's annotation queues) for dedicated human reviewers to score outputs against detailed rubrics, especially during development and testing.

Implementing Custom Evaluators

LangChain provides abstractions, often integrated with LangSmith, to streamline the implementation of custom evaluators. Typically, you define an evaluation function or class that takes the run information (inputs, outputs, etc.) and returns an EvaluationResult.

import re
from langsmith.evaluation import EvaluationResult, run_evaluator

# Example: Programmatic metric to check if output contains a specific warning phrase
@run_evaluator
def must_contain_warning(run, example) -> EvaluationResult:
    """Checks if the output contains 'Warning:'."""
    output = run.outputs.get("output") or ""
    if isinstance(output, str) and "Warning:" in output:
        score = 1 # Contains the warning
    else:
        score = 0 # Does not contain the warning
    return EvaluationResult(key="contains_warning", score=score)

# Example: Simplified semantic similarity check (conceptual)
# Assume `get_embedding` and `cosine_similarity` are defined elsewhere
@run_evaluator
def check_semantic_similarity(run, example) -> EvaluationResult:
    """Compares output embedding to reference answer embedding."""
    output = run.outputs.get("output") or ""
    reference = example.outputs.get("reference_answer") or ""

    if not output or not reference:
        return EvaluationResult(key="semantic_similarity", score=0, comment="Missing output or reference")

    output_embedding = get_embedding(output)
    reference_embedding = get_embedding(reference)
    similarity = cosine_similarity(output_embedding, reference_embedding) # Returns value between -1 and 1

    # Normalize score to 0-1 for consistency if desired
    normalized_score = (similarity + 1) / 2
    return EvaluationResult(key="semantic_similarity", score=normalized_score)

These evaluator functions can then be applied to datasets within LangSmith or used in custom evaluation scripts.

Different evaluation methods assess the generated output, producing scores and qualitative feedback.

Considerations When Defining Metrics

• Alignment with Goals: Metrics must directly reflect the application's objectives. A creative writing assistant needs different metrics than a factual Q&A system.
• Cost vs. Benefit: LLM-as-judge and human evaluation can be expensive or time-consuming. Balance the need for nuanced evaluation with available resources. Programmatic checks are cheaper but less comprehensive.
• Consistency and Reliability: LLM-as-judge evaluations can vary depending on the judge model and prompt. Ensure prompts are clear and test for consistency. Human evaluation requires clear rubrics and potentially multiple reviewers.
• Defining "Ground Truth": For generative tasks, a single "correct" answer might not exist. Use reference answers, golden datasets, or principles defined in rubrics as the basis for comparison.
• Iteration: Your understanding of what constitutes "good" performance will evolve. Revisit and refine your metrics as you gather more data and user feedback.

By thoughtfully defining custom metrics, you gain deeper insights into your LangChain application's performance, enabling targeted improvements and ensuring it meets the specific requirements of its production environment. These metrics form the bedrock for the automated evaluation pipelines and monitoring strategies discussed later in this chapter.