F1-Score: Combining Precision and Recall

Precision and recall are two important metrics for evaluating classification models. Precision measures how many of the positive predictions a model made were actually correct, while recall quantifies how many of the actual positive cases the model managed to identify. There is often an inherent trade-off between these metrics: improving precision can sometimes lower recall, and vice versa.

So, what if you need a single number that summarizes both? What if both finding all the relevant items (high recall) and ensuring the items found are relevant (high precision) are important for your application? This is where the F1-score comes in handy.

What is the F1-Score?

The F1-score is a way to combine precision and recall into a single metric. It's calculated as the harmonic mean of precision and recall.

Why Use the Harmonic Mean?

You might wonder, why not just take a simple average (arithmetic mean) of precision and recall? The harmonic mean has a useful property: it gives a lower weight to models where one metric is very high and the other is very low. A simple average might look good even if one metric is poor, but the harmonic mean penalizes this imbalance.

Consider a model with:

• Precision = 0.9 (90%)
• Recall = 0.1 (10%)

The arithmetic mean would be $(0.9 + 0.1) / 2 = 0.5$. This score doesn't seem too bad.

However, the F1-score (as we'll see how to calculate next) would be much lower, reflecting the poor recall. The harmonic mean pulls the combined score closer to the lower value, ensuring that a model must perform reasonably well on both precision and recall to achieve a high F1-score. It only scores high if both precision and recall are high.

Calculating the F1-Score

The formula for the F1-score is:

$$F1 = 2 \times \frac{Precision \times Recall}{Precision + Recall}$$

Let's use our previous example (Precision = 0.9, Recall = 0.1):

$$F1 = 2 \times \frac{0.9 \times 0.1}{0.9 + 0.1} = 2 \times \frac{0.09}{1.0} = 0.18$$

Notice how the F1-score (0.18) is much lower than the arithmetic mean (0.5) and closer to the lower metric (Recall = 0.1). This highlights the model's weakness in recall, which the simple average masked.

You can also express the F1-score directly using True Positives ($TP$), False Positives ($FP$), and False Negatives ($FN$):

$$F1 = \frac{2 \times TP}{2 \times TP + FP + FN}$$

This formula is derived by substituting the definitions of precision ($Precision = \frac{TP}{TP + FP}$) and recall ($Recall = \frac{TP}{TP + FN}$) into the first F1-score equation.

Interpreting the F1-Score

Like precision and recall, the F1-score ranges from 0 to 1.

• An F1-score of 1 represents perfect precision and recall.
• An F1-score of 0 means either precision or recall (or both) is zero.

A higher F1-score indicates a better balance between precision and recall. It's a particularly useful metric when:

1. Class imbalance is present: In datasets where one class significantly outnumbers the other(s), accuracy can be misleading. A high F1-score ensures the model is performing well on the minority class, considering both false positives and false negatives.
2. Both false positives and false negatives are important: If mistakenly identifying a negative case as positive ($FP$) and failing to identify a positive case ($FN$) are both undesirable outcomes, the F1-score provides a balanced assessment. For example, in medical diagnosis, both missing a disease (low recall) and incorrectly diagnosing a healthy patient (low precision) can have significant consequences.

Example Calculation

Let's revisit a confusion matrix:

	Predicted Positive	Predicted Negative
Actual Positive	TP = 80	FN = 20
Actual Negative	FP = 10	TN = 90

From this, we calculate:

• Precision = $\frac{TP}{TP + FP} = \frac{80}{80 + 10} = \frac{80}{90} \approx 0.89$
• Recall = $\frac{TP}{TP + FN} = \frac{80}{80 + 20} = \frac{80}{100} = 0.80$

Now, we calculate the F1-score:

$$F1 = 2 \times \frac{0.89 \times 0.80}{0.89 + 0.80} = 2 \times \frac{0.712}{1.69} \approx 2 \times 0.421 \approx 0.84$$

An F1-score of 0.84 suggests a good balance between precision (0.89) and recall (0.80).

In summary, the F1-score provides a single, convenient metric that summarizes a classifier's performance by balancing precision and recall. It's particularly valuable when dealing with imbalanced classes or when both types of classification errors (false positives and false negatives) need to be minimized.