Introduction to Set Theory for Probability

Experiments, outcomes, sample spaces (the set of all possible outcomes), and events (specific outcomes of interest) are fundamental concepts in probability. A very useful tool for working with these concepts is Set Theory. Set theory offers a formal way to group items together and describe relationships between those groups. In probability, the "items" are the outcomes of an experiment, and the "groups" are events.

An event is simply a set of outcomes from the sample space. If our sample space $S$ is rolling a standard six-sided die, $S = \{1, 2, 3, 4, 5, 6\}$. An event, like "rolling an even number," is the set $A = \{2, 4, 6\}$. Notice that $A$ is a collection of outcomes from $S$. We call $A$ a subset of $S$.

Set theory gives us a precise language and symbols to manipulate these events, which helps us calculate probabilities more complex than just single events. Let's look at the fundamental operations.

The Complement: What an Event Is Not

Sometimes, we're interested in the probability that an event doesn't happen. The complement of an event $A$, written as $A^c$ (or sometimes $A'$ or $\bar{A}$), represents all the outcomes in the sample space $S$ that are not in event $A$.

For example, if $S = \{1, 2, 3, 4, 5, 6\}$ and event $A$ is "rolling an even number", so $A = \{2, 4, 6\}$, then the complement $A^c$ is "not rolling an even number," which means "rolling an odd number."

$$A^c = \{1, 3, 5\}$$

Visually, if the entire box represents the sample space $S$, and the circle represents event $A$, the complement $A^c$ is everything outside the circle but still inside the box.

The shaded region represents the complement, $A^c$.

A fundamental rule in probability connects an event and its complement: The probability of an event happening plus the probability of it not happening must equal 1 (or 100%).

$$P(A) + P(A^c) = 1$$

This is often rearranged to find the probability of the complement:

$$P(A^c) = 1 - P(A)$$

This is incredibly useful. Sometimes it's much easier to calculate the probability that an event doesn't occur and subtract it from 1, rather than calculating the probability of the event directly.

The Intersection: When Events Overlap

What if we're interested in two (or more) events happening at the same time? This is where the intersection comes in. The intersection of two events $A$ and $B$, written as $A \cap B$, represents the set of outcomes that are present in both event $A$ and event $B$. Think of it as the "overlap" between the sets.

Let's use our die-rolling example again. $S = \{1, 2, 3, 4, 5, 6\}$. Let event $A$ be "rolling an even number": $A = \{2, 4, 6\}$. Let event $B$ be "rolling a number greater than 3": $B = \{4, 5, 6\}$.

The intersection, $A \cap B$, is the event "rolling an even number and rolling a number greater than 3". Which outcomes satisfy both conditions? Only 4 and 6.

$$A \cap B = \{4, 6\}$$

Visually, the intersection is the region where the circles representing events A and B overlap.

The overlapping green area represents the intersection, $A \cap B$.

The probability of the intersection, $P(A \cap B)$, is the probability that both events $A$ and $B$ occur. Calculating this depends on whether the events are independent or dependent, which we'll cover shortly.

The Union: When at Least One Event Occurs

What if we want to know the probability that at least one of two events happens? This involves the union. The union of two events $A$ and $B$, written as $A \cup B$, represents the set of outcomes that are in event $A$, or in event $B$, or in both.

Using our previous example: $A = \{2, 4, 6\}$ (even number) $B = \{4, 5, 6\}$ (greater than 3)

The union, $A \cup B$, is the event "rolling an even number or rolling a number greater than 3 (or both)". We combine all the unique elements from both sets:

$$A \cup B = \{2, 4, 5, 6\}$$

Notice that 4 and 6 appear in both sets, but we only list them once in the union.

Visually, the union covers all areas included in either circle A or circle B (including their overlap).

The entire shaded area (blue, yellow, and green overlap) represents the union, $A \cup B$.

Connecting Union to Probability: The Addition Rule

How do we calculate the probability of the union, $P(A \cup B)$? We might be tempted to just add $P(A)$ and $P(B)$. Let's try with our example: $P(A) = P(\{2, 4, 6\}) = 3/6 = 1/2$ $P(B) = P(\{4, 5, 6\}) = 3/6 = 1/2$

If we just add them: $P(A) + P(B) = 1/2 + 1/2 = 1$. But we found $A \cup B = \{2, 4, 5, 6\}$, so the actual probability is $P(A \cup B) = 4/6 = 2/3$. What went wrong?

We double-counted the outcomes that are in both sets (the intersection, $\{4, 6\}$). We added the probability of rolling a 4 and 6 when we calculated $P(A)$, and we added it again when we calculated $P(B)$.

To correct for this double-counting, we need to subtract the probability of the intersection:

$$P(A \cup B) = P(A) + P(B) - P(A \cap B)$$

This is known as the Addition Rule of Probability.

Let's apply it to our example: $A = \{2, 4, 6\}$, $P(A) = 3/6$ $B = \{4, 5, 6\}$, $P(B) = 3/6$ $A \cap B = \{4, 6\}$, $P(A \cap B) = 2/6$

So, $P(A \cup B) = P(A) + P(B) - P(A \cap B) = 3/6 + 3/6 - 2/6 = 4/6 = 2/3$. This matches our direct calculation!

Special Case: Mutually Exclusive Events If two events $A$ and $B$ cannot happen at the same time, they are called mutually exclusive (or disjoint). This means their intersection is empty ($A \cap B = \emptyset$), and therefore $P(A \cap B) = 0$.

For example, the event "rolling a 1" ($C=\{1\}$) and the event "rolling an even number" ($A=\{2, 4, 6\}$) are mutually exclusive. You can't roll a 1 and an even number on a single roll.

In this special case, the Addition Rule simplifies to:

$$P(A \cup B) = P(A) + P(B) \quad \text{(if A and B are mutually exclusive)}$$

Understanding complements, intersections, and unions provides the building blocks for analyzing more complex probabilistic scenarios, which are common when working with data and machine learning models. We'll build on these concepts as we explore conditional probability next.