Discrete Distribution: Bernoulli

The Bernoulli distribution represents the simplest discrete probability distribution. It is characterized by its Probability Mass Function (PMF) for discrete random variables.

Imagine an experiment that has only two possible outcomes. Think of flipping a coin (Heads or Tails), checking if an email is spam (Spam or Not Spam), or a user clicking an ad (Click or No Click). These are examples of Bernoulli trials.

A Bernoulli trial is a single random experiment with exactly two mutually exclusive outcomes, typically labeled as "success" and "failure".

The Bernoulli distribution models the probability of the outcome of a single Bernoulli trial. It depends on just one parameter:

• $p$: The probability of the outcome defined as "success".

Since there are only two outcomes, the probability of "failure" must be $1-p$.

Let's define a random variable $X$ that represents the outcome of a Bernoulli trial. Conventionally, we set:

• $X = 1$ if the outcome is a "success"
• $X = 0$ if the outcome is a "failure"

The Probability Mass Function (PMF) for a Bernoulli random variable $X$ is straightforward:

$P(X=1) = p$ $P(X=0) = 1-p$

This specifies the probability for each of the two possible values that $X$ can take. Sometimes, you might see this written more compactly using a single formula:

$P(X=k) = p^k (1-p)^{1-k} \quad \text{for } k \in \{0, 1\}$

Let's check if this compact formula works. If $k=1$ (success), the formula gives $p^1 (1-p)^{1-1} = p^1 (1-p)^0 = p \times 1 = p$. Correct. If $k=0$ (failure), the formula gives $p^0 (1-p)^{1-0} = p^0 (1-p)^1 = 1 \times (1-p) = 1-p$. Correct.

Examples of Bernoulli Trials

• Coin Flip: A single flip of a fair coin. Let "Heads" be success. Then $p=0.5$. $P(X=1) = 0.5$ and $P(X=0) = 0.5$. If the coin is biased and lands heads 70% of the time, then $p=0.7$. $P(X=1) = 0.7$ and $P(X=0) = 0.3$.
• Quality Control: Testing a single manufactured component. Let "Defective" be success (an unusual definition of success, but possible!). If 5% of components are defective, $p=0.05$. $P(X=1) = 0.05$ (probability of being defective) and $P(X=0) = 0.95$ (probability of not being defective).
• Medical Test: A patient takes a test for a specific condition. Let "Positive Test" be success. The probability $p$ would depend on the test's characteristics and whether the patient has the condition.

Visualizing the Bernoulli Distribution

Since there are only two outcomes, the visualization is simple. It's a bar chart with two bars. Let's visualize a Bernoulli distribution where the probability of success ($p$) is 0.7:

PMF for a Bernoulli distribution with $p=0.7$. The probability of failure ($X=0$) is $1-p=0.3$, and the probability of success ($X=1$) is $p=0.7$.

The Bernoulli distribution is fundamental because it's the building block for more complex distributions that deal with multiple trials, such as the Binomial distribution which we'll look at next. In machine learning, it often appears when modeling binary outcomes, like in logistic regression where the model predicts the probability of a positive class (a "success"). Understanding this simple distribution provides a solid base for exploring more involved probability concepts.