Calculating Prediction Errors

When a regression model makes a prediction, it differs from classification models that predict categories (like 'spam' or 'not spam'). Regression models predict continuous numbers. For example, a model built to predict tomorrow's temperature, or the price of a used car based on its features, provides a specific numerical prediction.

The fundamental question we need to answer is: How good are these numerical predictions? To do that, we first need to quantify the error for each individual prediction.

The Concept of Error (or Residual)

For every single data point you use to test your model, you have two values:

1. The Actual Value: This is the true, observed value. In our examples, it's the actual temperature that occurred or the actual selling price of the car. We often denote the actual value for the $i$-th data point as $y_i$.
2. The Predicted Value: This is the number your regression model generated for that same data point. We often denote the predicted value for the $i$-th data point as $\hat{y}_i$ (pronounced "y-hat").

The prediction error, often called the residual, is simply the difference between the actual value and the predicted value for a single observation.

Mathematically, we calculate the error $e_i$ for the $i$-th data point as:

$e_i = y_i - \hat{y}_i$

This formula tells us how far off the prediction was from the actual value.

An Example: Predicting Daily Temperatures

Let's say we're testing a model that predicts the maximum daily temperature (in degrees Celsius). We have the actual temperatures for four different days, and we also have the predictions our model made for those days:

• Day 1: Actual Temp ($y_1$) = 15°C, Predicted Temp ($\hat{y}_1$) = 16°C
• Day 2: Actual Temp ($y_2$) = 20°C, Predicted Temp ($\hat{y}_2$) = 19°C
• Day 3: Actual Temp ($y_3$) = 25°C, Predicted Temp ($\hat{y}_3$) = 23°C
• Day 4: Actual Temp ($y_4$) = 18°C, Predicted Temp ($\hat{y}_4$) = 18°C

Now, let's calculate the prediction error for each day using $e_i = y_i - \hat{y}_i$:

• Error for Day 1: $e_1 = 15 - 16 = -1$°C
• Error for Day 2: $e_2 = 20 - 19 = +1$°C
• Error for Day 3: $e_3 = 25 - 23 = +2$°C
• Error for Day 4: $e_4 = 18 - 18 = 0$°C

Interpreting the Errors

Notice the sign of the error:

• Negative Error ($e_1 = -1$): This means the predicted value (16°C) was higher than the actual value (15°C). The model overestimated the temperature.
• Positive Error ($e_2 = +1$, $e_3 = +2$): This means the predicted value (19°C, 23°C) was lower than the actual value (20°C, 25°C). The model underestimated the temperature.
• Zero Error ($e_4 = 0$): This means the predicted value (18°C) was exactly the same as the actual value (18°C). The model made a perfect prediction for this day.

Calculating these individual errors is the essential first step. Each error tells us the magnitude and direction of the inaccuracy for one specific prediction.

Scatter plot showing actual vs. predicted temperatures. The dashed line represents perfect predictions ($y = \hat{y}$). Arrows indicate the prediction error (vertical distance) for three data points.

While looking at individual errors is informative, especially for understanding specific failures of the model, it doesn't give us a single, overall measure of the model's performance across all predictions. If we have thousands or millions of predictions, we need a way to summarize these individual errors into one or more metrics.

These individual errors, $e_1, e_2, e_3, ...$, are the foundational components we'll use to calculate the aggregate regression metrics like Mean Absolute Error (MAE), Mean Squared Error (MSE), and Root Mean Squared Error (RMSE), which we will cover next.