Basic Probability Rules

1. The Rule of Completeness: Probability quantifies uncertainty. The probability of the entire sample space, the set of all possible outcomes, is always 1. For instance, when flipping a fair coin, the sample space consists of heads and tails. The probability of either outcome is 1 or 100%.

Probability of heads and tails for a fair coin

2. The Addition Rule for Mutually Exclusive Events: For events that cannot occur simultaneously, mutually exclusive events, the probability of either event is the sum of their individual probabilities. Rolling a six-sided die, the probability of rolling a 2 or a 5 is 1/6 + 1/6 = 1/3.

3. The General Addition Rule: For non-mutually exclusive events, the addition rule accounts for overlap. The probability of either event A or B is P(A) + P(B) - P(A and B). This avoids double-counting simultaneous events.

4. The Multiplication Rule for Independent Events: For independent events A and B, where one event doesn't affect the other's probability, P(A and B) = P(A) * P(B). Flipping two coins, the probability of both landing heads is 1/2 * 1/2 = 1/4.

Probability of both coins landing heads vs at least one tails

5. The General Multiplication Rule: When events aren't independent, the multiplication rule incorporates conditional probability, P(B|A), the probability of B given A has occurred. P(A and B) = P(A) * P(B|A). This rule is important for understanding event sequences where one influences another.

6. Conditional Probability and Bayes' Theorem: Conditional probability, P(B|A), represents the probability of B occurring after A has occurred. Bayes' Theorem, explored later, updates probabilities based on new information, critical for machine learning predictions and data learning.

7. The Law of Total Probability: This law calculates an event's probability by considering all possible mutually exclusive ways it can occur. It's useful for complex sample spaces and layered events, providing a systematic approach to breaking down probabilities.

These basic probability rules establish the foundation for more advanced topics, enabling you to address statistical challenges with confidence and precision. Keep these principles in mind as you study statistics and machine learning, where they will be essential in analyzing data and making informed predictions.