Common Split Ratios

To evaluate machine learning models effectively, data is typically split into a training set and a test set. The training set allows the model to learn, while the test set is reserved to assess how well the model performs on data it hasn't encountered before. This process helps estimate how the model will generalize to new, real situations.

But how much data should go into the training set versus the test set? There's no single magic number that works perfectly for every situation, but there are common conventions and guidelines that provide a good starting point. The choice often involves a trade-off:

1. More Training Data: Allows the model to potentially learn more patterns and complexities from the data, which might lead to a better-performing model. However, it leaves less data for testing.
2. More Test Data: Provides a more reliable estimate of the model's performance on unseen data. If your test set is very small, the performance metric you calculate might be due to luck (or bad luck!) rather than a true reflection of the model's capability.

Let's look at the most frequently used split ratios:

80/20 Split

This is perhaps the most common starting point. 80% of the data is used for training the model, and the remaining 20% is reserved for testing.

• Why it's popular: It provides a large chunk of data for the model to learn from while still reserving a reasonable amount for evaluation. For many moderately sized datasets, 20% is often large enough to give a fairly stable estimate of performance.

70/30 Split

Another widely used ratio allocates 70% of the data for training and 30% for testing.

• When to consider it: If you want a slightly more confident estimate of your model's generalization performance, increasing the test set size to 30% can help. This might be preferred if your dataset isn't massive, and you want to ensure the test set is adequately representative. It's also a good choice if the model you are training is relatively simple and doesn't require enormous amounts of data to learn effectively.

90/10 Split

With the rise of very large datasets (think millions or billions of examples), sometimes even 10% of the data is more than enough for a reliable test set.

• When to consider it: If you have a very large amount of data, using 90% for training allows the model to learn from the maximum possible information. The remaining 10%, while a smaller percentage, might still represent tens or hundreds of thousands of examples, which is plenty for getting a good performance estimate.

Common data split proportions visualized. The blue portion represents data used for training the model, and the orange portion represents data held out for testing.

Dataset Size Considerations

The total number of samples in your dataset heavily influences the appropriate split ratio:

• Small Datasets: If you have very limited data (say, only a few hundred examples), even a 20% test set might be too small to be reliable. In such cases, you might lean towards a 70/30 or even 60/40 split, accepting that your model won't have as much data to train on. Alternatively, this is where techniques like cross-validation, which we'll introduce briefly later, become especially important.
• Large Datasets: As mentioned, with abundant data, you can afford to allocate a smaller percentage (like 10% or even less) to testing, as the absolute number of test samples will still be large.

The Goal: Representative Sets

Regardless of the ratio, the primary goal is for both the training set and the test set to be representative of the overall data distribution. You want the patterns, variations, and potential challenges present in your full dataset to be reflected in both subsets. This is why simply taking the first 80% of your data is usually a bad idea, especially if the data has some inherent order (like time). Random shuffling before splitting, which we discuss next, is typically essential.

Choosing a split ratio is a practical decision. Starting with 80/20 or 70/30 is often reasonable. Consider your dataset size and how confident you need to be in your test results when making the final choice.