Moving an application from a development environment to production requires a focus on reliability, performance assessment, and ongoing health checks. Building sophisticated LangChain chains and agents is one part of the process; ensuring they consistently deliver correct results, operate within performance budgets, and can be diagnosed when problems occur is another critical aspect, especially given the variable nature of LLM outputs.

This chapter introduces the practices and tools necessary for the operational phase of LangChain applications. You will learn how to implement structured approaches for assessment, observation, and analysis. We will cover:

• Using LangSmith as a central platform for tracing execution, debugging issues, and monitoring behavior in production settings.
• Developing meaningful evaluation metrics specific to your application's goals.
• Creating automated evaluation pipelines to systematically test application outputs against defined datasets.
• Leveraging detailed traces to understand and resolve failures or unexpected behavior within chains and agents.
• Monitoring key operational indicators such as latency, error rates, token usage, and associated financial costs.
• Integrating application monitoring data with common third-party observability systems.
• Setting up mechanisms for collecting human feedback on application responses to guide improvements.

These techniques provide the foundation for maintaining dependable and effective LangChain applications once they are deployed.