Discrete Distribution: Binomial

Imagine you perform the same simple experiment multiple times, like flipping a coin. The Bernoulli distribution handles a single flip (one trial). But what if you flip the coin 10 times and want to know the probability of getting exactly 7 heads? This is where the Binomial distribution comes in.

The Binomial distribution models the number of "successes" in a fixed number of independent Bernoulli trials. A "success" is just the outcome we're interested in counting (like getting heads), and a "failure" is the other outcome (getting tails).

For a scenario to be modeled by a Binomial distribution, it must satisfy these conditions:

1. There are a fixed number of trials, denoted by $n$.
2. Each trial has only two possible outcomes: "success" or "failure".
3. The probability of success, denoted by $p$, is the same for each trial. The probability of failure is then $1-p$.
4. The trials are independent, meaning the outcome of one trial does not affect the outcome of another.

If these conditions hold, we can calculate the probability of getting exactly $k$ successes in $n$ trials.

The Binomial Probability Mass Function (PMF)

The Probability Mass Function (PMF) for a Binomial distribution gives the probability of observing exactly $k$ successes in $n$ trials. The formula is:

$P(X=k) = \binom{n}{k} p^k (1-p)^{n-k}$

Let's break down this formula:

• $X$ is the random variable representing the number of successes.
• $k$ is the specific number of successes we are interested in (where $k$ can be any integer from 0 to $n$).
• $n$ is the total number of trials.
• $p$ is the probability of success on a single trial.
• $(1-p)$ is the probability of failure on a single trial.
• $\binom{n}{k}$ is the binomial coefficient, read as "n choose k". It represents the number of different ways to arrange $k$ successes among $n$ trials. It's calculated as: $\binom{n}{k} = \frac{n!}{k!(n-k)!}$ where $n!$ (n factorial) is the product of all positive integers up to $n$ (e.g., $5! = 5 \times 4 \times 3 \times 2 \times 1 = 120$), and by definition, $0! = 1$.

The PMF formula combines these parts: $\binom{n}{k}$ counts the ways, $p^k$ gives the probability of those $k$ successes occurring, and $(1-p)^{n-k}$ gives the probability of the remaining $n-k$ failures occurring.

Example: Coin Flips

Let's go back to flipping a fair coin ($p=0.5$) 10 times ($n=10$). What's the probability of getting exactly 3 heads ($k=3$)?

1. Identify parameters: $n = 10$, $p = 0.5$, $k = 3$.
2. Calculate the binomial coefficient: $\binom{10}{3} = \frac{10!}{3!(10-3)!} = \frac{10!}{3!7!} = \frac{10 \times 9 \times 8}{3 \times 2 \times 1} = 120$ There are 120 different ways to get exactly 3 heads in 10 flips.
3. Calculate the probability part:
   • $p^k = (0.5)^3 = 0.125$
   • $(1-p)^{n-k} = (1-0.5)^{10-3} = (0.5)^7 \approx 0.0078125$
4. Combine: $P(X=3) = \binom{10}{3} p^3 (1-p)^{10-3} = 120 \times (0.5)^3 \times (0.5)^7 = 120 \times (0.5)^{10}$ $P(X=3) = 120 \times 0.0009765625 \approx 0.1172$

So, there's approximately an 11.72% chance of getting exactly 3 heads when flipping a fair coin 10 times.

Example: Quality Control

Suppose a factory produces light bulbs, and 5% of them are defective ($p=0.05$). If you randomly select 20 bulbs ($n=20$), what is the probability that exactly 1 bulb is defective ($k=1$)?

1. Identify parameters: $n = 20$, $p = 0.05$, $k = 1$.
2. Calculate the binomial coefficient: $\binom{20}{1} = \frac{20!}{1!(20-1)!} = \frac{20!}{1!19!} = \frac{20}{1} = 20$ There are 20 ways to choose which of the 20 bulbs is the single defective one.
3. Calculate the probability part:
   • $p^k = (0.05)^1 = 0.05$
   • $(1-p)^{n-k} = (1-0.05)^{20-1} = (0.95)^{19} \approx 0.3774$
4. Combine: $P(X=1) = \binom{20}{1} p^1 (1-p)^{20-1} = 20 \times (0.05)^1 \times (0.95)^{19}$ $P(X=1) \approx 20 \times 0.05 \times 0.3774 \approx 0.3774$

There's about a 37.74% chance that exactly one bulb in the sample of 20 is defective.

Visualizing the Binomial Distribution

We can visualize the probability of each possible outcome ($k=0, 1, ..., n$) using a bar chart representing the PMF. Here's the PMF for our coin flip example ($n=10, p=0.5$):

Binomial probability distribution for 10 trials with a success probability of 0.5 (e.g., flipping a fair coin 10 times). The most likely outcome is 5 successes.

Notice the symmetric shape when $p=0.5$. If $p$ were different (e.g., $p=0.2$), the distribution would be skewed.

Mean and Variance

Like other distributions, the Binomial distribution has measures of central tendency and spread:

• Mean (Expected Value): The average number of successes you'd expect over many repetitions of the $n$ trials. It's calculated simply as: $E[X] = \mu = np$ For $n=10, p=0.5$, the mean is $10 \times 0.5 = 5$. This matches the peak of the distribution shown above. For $n=20, p=0.05$, the mean is $20 \times 0.05 = 1$. We expect, on average, 1 defective bulb per sample of 20.
• Variance: A measure of how spread out the number of successes is likely to be around the mean. $Var(X) = \sigma^2 = np(1-p)$ For $n=10, p=0.5$, the variance is $10 \times 0.5 \times (1-0.5) = 2.5$. For $n=20, p=0.05$, the variance is $20 \times 0.05 \times (1-0.05) = 0.95$.
• Standard Deviation: The square root of the variance, giving a measure of spread in the original units. $\sigma = \sqrt{np(1-p)}$

Relevance in Machine Learning

The Binomial distribution is relevant in various machine learning scenarios:

• Classification Accuracy: If you test a model on $n$ independent data points, and the model has a probability $p$ of classifying a point correctly, the number of correct classifications can be modeled binomially.
• Click-Through Rates (CTR): In online advertising, if an ad is shown $n$ times (impressions) and has a probability $p$ of being clicked each time (assuming independence), the total number of clicks follows a Binomial distribution.
• A/B Testing: Comparing the success rates (e.g., conversion rates) of two different versions (A and B) often involves analyzing Binomial outcomes.

Understanding the Binomial distribution helps in modeling count data for binary outcomes, setting expectations, and evaluating performance where repeated independent trials occur.