Accuracy: A Simple First Metric

A fundamental question when evaluating classification models is: 'How often does the model get the prediction right?' Accuracy directly addresses this question, serving as a simple metric for evaluating classification performance.

What is Accuracy?

Accuracy measures the overall correctness of the model. It tells us the proportion of predictions that the model classified correctly out of the total number of predictions made. Think of it as the hit rate of your model. If a model has an accuracy of 0.9, it means it correctly predicted the class label for 90% of the instances it saw.

Calculating Accuracy

The calculation for accuracy is straightforward. You simply count the number of predictions your model got right and divide that by the total number of predictions it made.

The formula is:

$$\text{Accuracy} = \frac{\text{Number of Correct Predictions}}{\text{Total Number of Predictions}}$$

A "correct prediction" happens when the predicted class label matches the actual class label for a given data point. The "total number of predictions" is typically the total number of data points in your test set (the data you are using to evaluate the model).

A Simple Example

Imagine we have a simple classification model designed to predict whether an email is 'Spam' or 'Not Spam'. We test this model on a set of 10 emails it hasn't seen before. Here are the results:

Actual Label	Predicted Label	Correct?
Spam	Spam	Yes
Not Spam	Not Spam	Yes
Spam	Spam	Yes
Spam	Not Spam	No
Not Spam	Not Spam	Yes
Spam	Spam	Yes
Not Spam	Spam	No
Not Spam	Not Spam	Yes
Spam	Spam	Yes
Spam	Spam	Yes

Let's count:

• Total Number of Predictions: We evaluated 10 emails.
• Number of Correct Predictions: Looking at the 'Correct?' column, we count 8 'Yes' entries.

Now, we apply the formula:

$$\text{Accuracy} = \frac{8}{10} = 0.8$$

So, the accuracy of our email spam filter model on this test set is 0.8, or 80%. This means the model correctly classified 8 out of the 10 emails.

We can visualize this simple count:

Counts of correct and incorrect predictions for the email spam example.

Interpreting Accuracy

Accuracy scores range from 0 (meaning the model got every prediction wrong) to 1 (meaning the model got every prediction right). Higher accuracy generally indicates a better-performing model. It gives a quick, overall summary of how the model is doing.

Because it's easy to understand and calculate, accuracy is often the first metric people look at. However, while useful, accuracy doesn't always provide a complete picture of a model's performance. In the next section, we'll explore situations where relying solely on accuracy can be misleading.