You have learned numerous techniques to optimize machine learning models through sophisticated compiler and runtime strategies. However, applying these optimizations is only part of the process. Measuring their impact and identifying residual bottlenecks are necessary steps to ensure maximum performance. The complex transformations performed by compilers can often make it difficult to understand the performance characteristics of the final executed code.

This chapter focuses on the tools and methodologies required for effective performance analysis of compiled ML workloads. We will cover:

• The specific difficulties encountered when profiling code generated by ML compilers.
• Using system-level profilers (CPU, GPU, interconnect) to get a broad view of execution.
• Employing specialized tools like Intel VTune, NVIDIA Nsight, and AMD ROCprof for detailed CPU and GPU kernel analysis.
• Techniques to correlate high-level framework operations with the specific low-level compiled kernels being executed.
• Methods for analyzing memory access patterns and interpreting compiler optimization reports.

By the end of this chapter, you will be equipped to use advanced profiling tools to diagnose performance issues in compiled ML code, guiding further optimization efforts.