Interval Estimation: Confidence Intervals

A point estimate, like the sample mean $\bar{x}$, gives us a single number as our best guess for an unknown population parameter, like the population mean $\mu$. For instance, if we calculate the average height from a sample of people to be 175 cm, that's our point estimate for the average height of all people in the population we're interested in.

But how much faith should we put in that single number? If we took a different sample, we'd likely get a slightly different sample mean, maybe 173 cm or 176 cm. A point estimate doesn't tell us anything about this uncertainty or how much the estimate might vary from sample to sample. This is where interval estimation comes in.

Instead of providing just one number, an interval estimate gives us a range of plausible values for the population parameter. This range is called a confidence interval.

What is a Confidence Interval?

A confidence interval is a range calculated from sample data that is likely to contain the true value of an unknown population parameter. It's defined by two numbers: a lower bound and an upper bound. For example, instead of just saying the average height is 175 cm, we might say we are "95% confident that the true average height of the population is between 172 cm and 178 cm."

This interval [172 cm, 178 cm] is the confidence interval. The "95%" part is the confidence level.

Understanding the Confidence Level

The confidence level (commonly 90%, 95%, or 99%) represents how confident we are in the process used to generate the interval. It's important to interpret this correctly:

• Correct Interpretation: If we were to repeat our sampling process many, many times and construct a 95% confidence interval for each sample, we would expect about 95% of those intervals to capture the true, fixed (but unknown) population parameter.
• Common Misinterpretation: It does not mean that there is a 95% probability that the true population parameter lies within one specific calculated interval. Once we have calculated an interval from our sample (e.g., 172 cm to 178 cm), the true population parameter either is or is not within that specific range. It doesn't jump in and out. The uncertainty lies in whether the particular sample we drew happened to yield an interval that includes the true value.

Think of it like trying to toss rings onto a fixed peg (the true population parameter). The confidence level tells you the success rate of your ring-tossing method. If you have a 95% success rate, it means that over many tosses, 95% of your rings will land around the peg. For any single toss (any single calculated interval), the ring either landed on the peg or it didn't.

The true population mean (dashed red line) is fixed but unknown. Each vertical line represents a 95% confidence interval calculated from a different random sample. Most intervals (gray lines) contain the true mean, but occasionally (like sample 16) an interval misses it due to random sampling variation. With a 95% confidence level, we expect about 1 out of 20 intervals to miss the true value purely by chance.

What Influences the Width of a Confidence Interval?

The width of the confidence interval gives us a sense of the precision of our estimate. A narrower interval suggests a more precise estimate, while a wider interval indicates more uncertainty. Three main factors influence the width:

1. Confidence Level: If we want higher confidence (e.g., 99% instead of 95%), we need to cast a wider net to be more certain of capturing the true parameter. So, a higher confidence level leads to a wider interval.
2. Sample Size ($n$): As we collect more data (increase the sample size), our estimate becomes more reliable, and the uncertainty decreases. Therefore, a larger sample size leads to a narrower interval.
3. Variability in the Data: If the data points themselves are highly spread out (high standard deviation), it's harder to pinpoint the population parameter accurately. More variability in the sample data leads to a wider interval.

Confidence intervals provide a much richer picture than point estimates alone. They give us a plausible range for the true parameter value and explicitly communicate the level of uncertainty associated with our sample-based estimate. This concept is fundamental for interpreting results not just in statistics, but also when evaluating the reliability of performance metrics for machine learning models trained or tested on sample data.