Having examined specific metrics for classification and regression, along with the rationale for data splitting, we now integrate these concepts into a practical workflow. This chapter details a standard sequence for evaluating machine learning models.
You will learn how to structure the evaluation process, starting from choosing relevant metrics for your specific problem type, correctly partitioning your data into training and testing subsets, generating predictions using your model on the unseen test data, and finally calculating and interpreting the resulting performance metrics. This provides a structured method to determine how effectively a model performs on new, previously unobserved examples. We will also address frequent errors encountered during this process.
5.1 Steps in a Standard Evaluation
5.2 Choosing Metrics for Your Problem
5.3 Performing the Train-Test Split
5.4 Training a Simple Model
5.5 Generating Predictions on the Test Set
5.6 Calculating Performance Metrics
5.7 Interpreting the Results
5.8 Simple Evaluation Workflow Example
5.9 Common Mistakes in Basic Evaluation