ApX logoApX logo
ModelsCalculatorHardwareCoursesPricing
ApX logoApX logo
ModelsCalculatorHardwareCoursesPricing

All Courses

Probability & Statistics Fundamentals for Machine Learning
Chapter 1: Introduction to Probability and Statistics for Machine Learning
What are Probability and Statistics?
Relevance in Machine Learning
Understanding Data Types
Populations and Samples
Setting Up the Python Environment
Hands-on Practical: Loading and Inspecting Data
Quiz for Chapter 1
Chapter 2: Descriptive Statistics: Summarizing Data
Measuring the Center: Mean, Median, and Mode
Measuring Spread: Range
Measuring Spread: Variance and Standard Deviation
Understanding Percentiles and Quartiles
Visualizing Distributions: Histograms
Visualizing Summaries: Box Plots
Calculating Descriptive Statistics with Python
Practice: Summarizing a Dataset
Quiz for Chapter 2
Chapter 3: Basic Probability Concepts
Understanding Probability: Events and Sample Spaces
Calculating Simple Probabilities
Introduction to Set Theory for Probability
Conditional Probability Explained
Independent vs. Dependent Events
Introduction to Bayes' Theorem
Practice: Probability Calculations
Quiz for Chapter 3
Chapter 4: Probability Distributions
What are Probability Distributions?
Probability Mass Function (PMF) for Discrete Distributions
Discrete Distribution: Bernoulli
Discrete Distribution: Binomial
Probability Density Function (PDF) for Continuous Distributions
Continuous Distribution: Uniform
Continuous Distribution: Normal (Gaussian)
The Central Limit Theorem
Generating Samples from Distributions using Python
Practice: Working with Distributions
Quiz for Chapter 4
Chapter 5: Introduction to Statistical Inference
Drawing Conclusions from Data
Point Estimation
Interval Estimation: Confidence Intervals
Hypothesis Testing: The Basic Idea
Null and Alternative Hypotheses
Understanding P-values
Connecting Inference to Machine Learning Evaluation
Practice: Interpreting Statistical Results
Quiz for Chapter 5

Measuring Spread: Variance and Standard Deviation

Knowing the center of your data with the mean, median, or mode gives you a valuable reference point. However, it doesn't tell the full story. For example, take two sets of exam scores: both might have a mean score of 75, but in one set, scores might cluster tightly between 70 and 80, while in the other, they might range wildly from 40 to 100. Measures of central tendency alone don't capture this difference in spread or variability.

To quantify how spread out the data points are, we use measures of dispersion. The most common and important measures are variance and standard deviation. They tell us how much, on average, the data points deviate from their mean.

Variance: The Average Squared Difference

Imagine you have a dataset and you've calculated its mean (μ\muμ for a population, xˉ\bar{x}xˉ for a sample). A natural first step to measure spread might be to calculate each data point's deviation from the mean (xi−μx_i - \muxi​−μ or xi−xˉx_i - \bar{x}xi​−xˉ) and average these deviations. However, there's a problem: the positive deviations (points above the mean) and negative deviations (points below the mean) will always cancel each other out, resulting in an average deviation of zero. This isn't very informative!

To overcome this cancellation, we square each deviation before averaging them. Squaring makes all deviations positive, ensuring they don't cancel out. This average of the squared deviations is called the variance.

Population Variance (σ2\sigma^2σ2)

If your dataset represents the entire population of interest, the variance (denoted by σ2\sigma^2σ2, the Greek letter sigma squared) is calculated as:

σ2=∑i=1N(xi−μ)2N\sigma^2 = \frac{\sum_{i=1}^{N} (x_i - \mu)^2}{N}σ2=N∑i=1N​(xi​−μ)2​

Where:

  • NNN is the total number of data points in the population.
  • xix_ixi​ is each individual data point.
  • μ\muμ is the population mean.
  • ∑\sum∑ shows summing up the squared differences for all data points from i=1i=1i=1 to NNN.

Sample Variance (s2s^2s2)

More often, we work with a sample, a subset of the population. When estimating the population variance from a sample, we use a slightly different formula for the sample variance (denoted by s2s^2s2):

s2=∑i=1n(xi−xˉ)2n−1s^2 = \frac{\sum_{i=1}^{n} (x_i - \bar{x})^2}{n-1}s2=n−1∑i=1n​(xi​−xˉ)2​

Where:

  • nnn is the number of data points in the sample.
  • xix_ixi​ is each individual data point in the sample.
  • xˉ\bar{x}xˉ is the sample mean.
  • ∑\sum∑ shows summing up the squared differences for all data points from i=1i=1i=1 to nnn.

Notice the denominator is n−1n-1n−1 instead of nnn. This is known as Bessel's correction. Using n−1n-1n−1 makes the sample variance an unbiased estimator of the population variance, meaning that if you took many samples and calculated their variances using n−1n-1n−1, the average of those sample variances would be closer to the true population variance. Intuitively, the sample mean xˉ\bar{x}xˉ is calculated from the sample itself, making the sample data points slightly closer to xˉ\bar{x}xˉ on average than they would be to the true population mean μ\muμ. Dividing by a smaller number (n−1n-1n−1) slightly inflates the variance estimate, counteracting this effect. For a beginner level, the main takeaway is that standard statistical software and libraries use n−1n-1n−1 when calculating variance from sample data.

A larger variance indicates that the data points are, on average, farther away from the mean (more spread out). A smaller variance indicates they are clustered more closely around the mean.

One drawback of variance is its units. If your data represents heights in centimeters (cm), the variance will be in square centimeters (cm²), which is difficult to interpret directly in the context of the original data. This leads us to the standard deviation.

Standard Deviation: Back to Original Units

The standard deviation is simply the square root of the variance. It's often preferred over variance because it brings the measure of spread back into the same units as the original data, making it much more interpretable.

Population Standard Deviation (σ\sigmaσ)

For a population, the standard deviation (denoted by σ\sigmaσ) is:

σ=σ2=∑i=1N(xi−μ)2N\sigma = \sqrt{\sigma^2} = \sqrt{\frac{\sum_{i=1}^{N} (x_i - \mu)^2}{N}}σ=σ2​=N∑i=1N​(xi​−μ)2​​

Sample Standard Deviation (sss)

For a sample, the standard deviation (denoted by sss) is:

s=s2=∑i=1n(xi−xˉ)2n−1s = \sqrt{s^2} = \sqrt{\frac{\sum_{i=1}^{n} (x_i - \bar{x})^2}{n-1}}s=s2​=n−1∑i=1n​(xi​−xˉ)2​​

The standard deviation provides a measure of the typical or average distance of data points from the mean.

  • A low standard deviation means the data points tend to be very close to the mean. The distribution curve (like a histogram) will be tall and narrow.
  • A high standard deviation means the data points are spread out over a wider range of values. The distribution curve will be shorter and wider.

Let's look at two datasets with the same mean but different standard deviations.

Both datasets might have a mean around 75. Dataset A (blue) has values clustered tightly, resulting in a low standard deviation. Dataset B (pink) has values spread much wider, resulting in a high standard deviation.

Understanding variance and standard deviation is fundamental in statistics and machine learning. They are used in:

  • Data exploration: Quantifying the variability in features.
  • Feature scaling: Techniques like standardization use the mean and standard deviation to rescale features, which can be important for algorithms sensitive to feature magnitudes (like support vector machines or principal component analysis).
  • Statistical inference: Estimating confidence intervals and performing hypothesis tests often involves the standard deviation.
  • Anomaly detection: Points falling many standard deviations away from the mean are outliers.

While the range gives a quick sense of the total spread, variance and standard deviation provide a more detailed and widely used measure of how data clusters around the central value. You'll learn how to calculate these efficiently using Python in the "Calculating Descriptive Statistics with Python" section later in this chapter.

  • Probability & Statistics for Engineers & Scientists, Ronald E. Walpole, Raymond H. Myers, Sharon L. Myers, Keying E. Ye, 2016 (Pearson Education) - This is a classic and comprehensive textbook that provides a rigorous yet accessible treatment of descriptive statistics, including the theoretical basis and calculation of variance and standard deviation.
  • Practical Statistics for Data Scientists: 50 Essential Concepts, Peter Bruce, Andrew Bruce, Peter Gedeck, 2020 (O'Reilly Media) - This book offers a practical, applied perspective on statistical concepts for machine learning practitioners, clearly explaining variance and standard deviation within a data science context.
  • Introduction to Probability and Statistics (18.05), Jeremy Orloff, Jonathan Bloom, 2014 (MIT OpenCourseWare) - A free, high-quality university course that covers fundamental statistical measures, providing video lectures, notes, and problem sets for a deeper understanding.

© 2026 ApX Machine Learning