Evaluation Strategies: Metrics and Human Feedback

Evaluating the performance of LLM applications means more than checking if the code runs without errors. Since LLMs generate human-like text, their output is often probabilistic and can vary even for the same input. This inherent variability means traditional deterministic tests aren't enough. We need strategies to assess the quality, accuracy, and usefulness of the generated content, which often involves a blend of automated calculations and human judgment.

Quantitative Metrics

Automated metrics provide a scalable way to get objective, numerical scores for certain aspects of LLM performance. They are particularly useful during development for comparing different model versions, prompt strategies, or retrieval methods. However, remember that these metrics are often proxies for true quality and have limitations.

Information Retrieval Metrics (Especially for RAG)

When your application involves retrieving information to ground the LLM (like in Retrieval-Augmented Generation), you can evaluate the retrieval step itself using standard IR metrics:

Precision: Out of the documents retrieved, what fraction is relevant? $Precision = \frac{Number \, of \, Relevant \, Retrieved \, Documents}{Total \, Number \, of \, Retrieved \, Documents}$
Recall: Out of all possible relevant documents, what fraction did the system retrieve? $Recall = \frac{Number \, of \, Relevant \, Retrieved \, Documents}{Total \, Number \, of \, Relevant \, Documents}$
F1-Score: The harmonic mean of Precision and Recall, providing a single score balancing both. $F1 = 2 \times \frac{Precision \times Recall}{Precision + Recall}$
Mean Reciprocal Rank (MRR): Measures the average rank of the first relevant document retrieved across multiple queries. Useful when you primarily care about finding at least one good result quickly.
Normalized Discounted Cumulative Gain (NDCG): Evaluates the quality of the ranking of retrieved documents, giving higher scores for relevant documents appearing earlier in the list.

These metrics require ground truth data, meaning you need pre-defined sets of relevant documents for your test queries.

Text Similarity Metrics

These metrics compare the LLM's generated text against one or more reference texts (often human-written examples of good answers).

BLEU (Bilingual Evaluation Understudy): Measures the overlap of n-grams (sequences of words) between the generated text and reference texts. It penalizes outputs that are too short. Primarily used in machine translation.
ROUGE (Recall-Oriented Understudy for Gisting Evaluation): Measures overlap based on n-grams (ROUGE-N), longest common subsequences (ROUGE-L), or skip-bigrams. Often used for evaluating summaries.
METEOR (Metric for Evaluation of Translation with Explicit ORdering): Considers exact word matches, stemmed matches, and synonym matches, aligning generated and reference texts. It includes a penalty for incorrect word order.

While easy to compute, these lexical overlap metrics often fail to capture semantic meaning. An output could use different words but mean the same thing (low score) or use similar words but be nonsensical (high score).

Semantic Similarity Metrics

To address the limitations of lexical overlap, semantic similarity metrics use text embeddings (numerical representations capturing meaning).

Cosine Similarity: Calculate embeddings for the generated text and the reference text (using models like Sentence-BERT). The cosine similarity between these embedding vectors ( $cos(\theta) = \frac{A \cdot B}{\|A\| \|B\|}$ ) measures how closely aligned they are in meaning. A score closer to 1 indicates higher semantic similarity.

This approach is better at understanding paraphrasing and synonyms but still doesn't guarantee factual accuracy or coherence.

Task-Specific Metrics

Depending on your application's goal, you might use more direct metrics:

Accuracy: For classification tasks (e.g., sentiment analysis, intent recognition), what percentage of predictions are correct?
Exact Match (EM): For question answering, does the generated answer exactly match the reference answer? This is very strict.
Factual Consistency: Does the generated text contradict known facts or information present in a provided source document (relevant for RAG)? Measuring this often requires more sophisticated techniques or human review.

Human Evaluation

Despite the utility of automated metrics, human judgment remains the gold standard for assessing many aspects of LLM output quality that metrics struggle with:

Fluency and Coherence: Does the text read naturally? Is it logically structured?
Factual Accuracy and Hallucinations: Is the information correct? Does the model invent facts?
Relevance and Helpfulness: Does the response actually address the user's query or goal? Is it useful?
Tone and Style: Does the output match the desired persona or style guide?
Safety and Bias: Does the output contain harmful, biased, or inappropriate content?
Instruction Following: Did the LLM adhere to specific constraints or instructions in the prompt?

Common approaches for collecting human feedback include:

Likert Scales: Raters score responses on numerical scales (e.g., 1-5) for specific attributes like "Accuracy," "Fluency," or "Helpfulness." Clear rubrics are essential for consistency.
Pairwise Comparison: Raters are shown two responses (e.g., from Model A vs. Model B, or before vs. after a change) and asked to choose which one is better according to specific criteria. This is often easier and more reliable than assigning absolute scores.
Ranking: Raters rank multiple responses from best to worst.
Annotation and Error Analysis: Raters identify and categorize specific errors within the generated text (e.g., marking factual errors, grammatical mistakes, refusals). This provides detailed qualitative feedback for improvement.

Integrated evaluation workflow combining automated metrics for speed and scale during development with human evaluation for quality assessment and validation.

Human evaluation is resource-intensive (time and cost) and can suffer from subjectivity. Establishing clear guidelines, training raters, and measuring inter-annotator agreement (how consistently different raters apply the guidelines) are important steps to ensure reliable results.

Combining Metrics and Human Feedback

The most effective evaluation strategy typically involves using both quantitative metrics and human feedback:

Use Metrics for Scale: Employ automated metrics during development for rapid feedback on changes, A/B testing different prompts or models, and monitoring for regressions. They help you iterate quickly.
Use Humans for Depth and Ground Truth: Rely on human evaluation for assessing metrics can't capture, understanding why outputs are good or bad, identifying errors, validating that your automated metrics correlate with actual perceived quality, and setting quality benchmarks.
Iterative Refinement: Use insights from human evaluation to refine your prompts, fine-tune models, improve retrieval strategies, and potentially even develop better, custom automated metrics that more closely reflect the qualities you care about for your specific application.

By thoughtfully combining these approaches, you can gain a comprehensive understanding of your LLM application's performance and systematically improve its quality and reliability. Frameworks discussed later can help operationalize these evaluation strategies.

Build LLM apps faster with Kerb

Cleaner syntax. Built-in debugging. Production-ready from day one.

Built for the AI systems behind ApX Machine Learning

Was this section helpful?

References

Introduction to Information Retrieval, Christopher D. Manning, Prabhakar Raghavan, and Hinrich Schütze, 2008 (Cambridge University Press) - A foundational textbook covering core information retrieval concepts, including standard evaluation metrics like Precision, Recall, F1-Score, MRR, and NDCG.
BLEU: a Method for Automatic Evaluation of Machine Translation, Kishore Papineni, Salim Roukos, Todd Ward, Wei-Jing Zhu, 2002 Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics (Association for Computational Linguistics) DOI: 10.3115/1073083.1073135 - Introduces BLEU, a widely used metric for evaluating the quality of machine-translated text by comparing it to reference translations.
Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks, Nils Reimers and Iryna Gurevych, 2019 Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) (Association for Computational Linguistics) DOI: 10.18653/v1/D19-1410 - Presents Sentence-BERT, a modification of BERT that yields semantically meaningful sentence embeddings useful for tasks like semantic similarity and clustering.
Holistic Evaluation of Language Models, Percy Liang, Rishi Bommasani, Tony Lee, Dimitris Tsipras, Dilara Soylu, et al., 2023 Transactions on Machine Learning Research (TMLR) DOI: 10.48550/arXiv.2211.09110 - Proposes a broad framework for evaluating language models across diverse metrics, scenarios, and human preferences, addressing limitations of traditional metrics.