What is Probability?

Probability forms the bedrock of statistics and machine learning, enabling us to quantify uncertainty and make informed predictions based on data. It provides a framework for understanding the likelihood of events occurring and managing inherent uncertainties in real-world scenarios.

To understand probability, consider an "experiment" – any process or action resulting in one or more outcomes. Flipping a coin, rolling a die, or drawing a card are experiments. Each possible result is an "outcome." For a coin flip, the outcomes are "heads" or "tails." For a six-sided die, the outcomes are the numbers 1 through 6.

The set of all possible outcomes is the "sample space." For a coin flip, the sample space is {heads, tails}, and for a die roll, it is {1, 2, 3, 4, 5, 6}. An "event" is any subset of the sample space. For example, obtaining an even number when rolling a die is an event consisting of the outcomes {2, 4, 6}.

Probability of even and odd outcomes when rolling a die

The probability of an event is a numerical value between 0 and 1 that reflects its likelihood of occurrence. A probability of 0 indicates impossibility, while 1 denotes certainty. For example, the probability of rolling a number greater than 0 on a standard die is 1, as all outcomes are greater than 0. Conversely, the probability of rolling a 7 is 0, since 7 is not a possible outcome.

Several interpretations help us understand how probabilities are assigned:

1. Classical Probability: Assumes all outcomes in a sample space are equally likely. It's often used in simple scenarios, like flipping a fair coin or rolling a fair die. The probability of an event is calculated by dividing the number of favorable outcomes by the total number of possible outcomes. For example, the probability of rolling a 4 on a fair die is 1/6, as there is one favorable outcome (rolling a 4) out of six possible outcomes.

{"type":"bar","data":{"labels":["1","2","3","4","5","6"],"datasets":[{"label":"Probability","data":[1/6,1/6,1/6,1/6,1/6,1/6],"backgroundColor":["#74b816","#74b816","#74b816","#74b816","#74b816","#74b816"]}]}}

Probability of each outcome when rolling a fair die

2. Frequentist Probability: Defines probability as the long-run relative frequency of an event occurring in repeated trials of an experiment. It's empirical and based on observations. For example, if you flip a coin 1000 times and it lands on heads 510 times, the frequentist probability of getting heads is 510/1000, or 0.51.

3. Subjective Probability: Considers probability as a measure of personal belief or degree of certainty about an event's occurrence. It's often used when there's no clear empirical basis to calculate probability, relying instead on expert judgment or intuition. For example, a weather forecaster might assign a subjective probability of 0.7 to the event of rain tomorrow based on current weather patterns and experience.

The addition rule is used to find the probability of either of two mutually exclusive events occurring. For instance, if you want to know the probability of rolling either a 2 or a 5 on a die, you can add the probabilities of each event: $P(2 \text{ or } 5) = P(2) + P(5) = 1/6 + 1/6 = 1/3$.

The multiplication rule comes into play when we're interested in the probability of two independent events both occurring. For example, if we roll two dice, the probability of both showing a 6 is the product of their individual probabilities: $P(\text{both 6}) = P(6 \text{ on first die}) \times P(6 \text{ on second die}) = 1/6 \times 1/6 = 1/36$.

Conditional probability, denoted as $P(A|B)$, is the probability of an event A occurring given that another event B has already occurred. Understanding conditional probability is crucial for analyzing dependent events and is a foundational concept in many machine learning algorithms.

Finally, we explore independence. Two events are independent if the occurrence of one does not affect the probability of the other. For example, flipping a coin and rolling a die are independent events, as the outcome of the coin flip does not influence the die roll.

As we progress, these fundamental concepts will serve as building blocks for more advanced topics. By understanding probability, you'll gain the tools to navigate uncertainty with confidence, paving the way for more sophisticated data analysis and machine learning techniques.