Having explored methods for aligning Large Language Models, the next logical step is to determine how effectively these methods work and whether the resulting models are genuinely safe. Building an LLM is one part of the process; rigorously assessing its behavior is another critical component. Without robust evaluation, claims of alignment and safety remain unsubstantiated.

This chapter provides the techniques and frameworks needed for this assessment. We will cover:

• Defining and operationalizing key safety dimensions, often summarized as Helpfulness, Honesty, and Harmlessness (HHH).
• Using automated benchmarks like HELM and TruthfulQA to measure specific capabilities and failure modes.
• Implementing structured human evaluation protocols and red teaming strategies to uncover subtle issues missed by automated tests.
• Introducing metrics and methods for quantifying bias and fairness in model outputs.
• Assessing how models perform under distributional shifts, a key factor for real-application reliability.
• Discussing the practical difficulties in achieving scalable and consistently reliable evaluation.

By the end of this chapter, you will understand how to apply a multi-faceted approach to evaluate the alignment and safety characteristics of LLMs, moving beyond anecdotal checks to more systematic analysis.