Previously on Probability…

• Basic probability rules

• Independence

• DeMorgan’s laws

Independence vs Dependence

• Independence vs Dependence

• Take-home message: Understanding conditional probability

Rolling a Fair Six-Sided Die

Consider another classic problem in probability: rolling a die with six sides, numbered from 1 to 6.

• Sample Space: \(\Omega = \{1,2,3,4,5,6\}\)

• Probability of each outcome is \(P(x_i) = \frac{1}{6}\) where \(x_i\) is an outcome in the \(i\)th roll.

3.33-Minute Activity

• Question - What is the probability of rolling a six-sided die 5 times and seeing: \(\{1,6,4,5,1\}\)?

• Timer starts.

• Answer: \[P(x_1,x_2,x_3,x_4,x_5) = \frac{1}{6}\frac{1}{6}\frac{1}{6}\frac{1}{6}\frac{1}{6} \approx 0.0001286\]

• Why? Because each roll is independent and each outcome in the sample space is equally likely to occur.

3.33-Minute Activity

• Question - What is the probability that of getting a 6 for the first time on the 5th roll?

• Timer starts.

• Step 1: The sample space is \(6^5 = 7776\).

• Step 2: Each throw is independent. First 4 rolls can be 1, 2, 3, 4, or 5,with probability \(\frac{5}{6}\) for each roll. Last roll must be a 6 with probability \(\frac{1}{6}\). \[P(\text{get 6 for the 1st time on the 5th roll}) = \left(\frac{5}{6}\right)^4 \left(\frac{1}{6}\right)^1 = \left(\frac{5^4}{6^5}\right) \approx 0.0804\]

How many rolls - on average - do you need to observe the 1st 6?

\[ \begin{align} P(\text{1 roll until 1st 6}) & = \frac{1}{6} \\ P(\text{2 rolls until 1st 6}) & = \frac{5}{6}\frac{1}{6} = \frac{5}{6^2} \\ & \vdots \\ P(\text{n rolls until 1st 6}) & = \frac{5^{n-1}}{6^n} \end{align} \]

• Question - What is the event space? Infinitely many event outcomes but not equally likely: \(\{1,2,3,4,5, ...\}\).

Probability Urns

• Classical problem in probability and statistics.

• Purpose: We quote from Wiki: Urn Problem

“In probability and statistics, an urn problem is an idealized mental exercise in which some objects of real interest (such as atoms, people, cars, etc.) are represented as colored balls in an urn or other container. One pretends to draw (remove) one or more balls from the urn; the goal is to determine the probability of drawing one color or another, or some other properties. A key parameter is whether each ball is returned to the urn after each draw.”

Independence

• Suppose I have an urn with only 2 balls, 1 black and 1 red.

• If I sample with replacement, this is equivalent to flipping a fair coin: \(P(Black) = \frac{1}{2}\) and \(P(Red) = \frac{1}{2}\).

• Question - What about 10 balls, 5 black and 5 red?

• The probability is the same!

• Sampling with replacement = independence

3.33-Minute Activity

• Question - What about 10 balls, 7 black and 3 red? What is the probability that sampling with replacement I draw: red, red, black?

• Timer starts.

• If I sample with replacement, this is equivalent to flipping a biased coin: \(P(Black) = \frac{7}{10}\) and \(P(Red) = \frac{3}{10}\).

• What is the probability that sampling with replacement I draw: red, red, black \[P(red,red,black) = \left(\frac{3}{10}\right)^2 \left(\frac{7}{10}\right) = \frac{63}{1000}\]

Dependence

• Now think about sampling without replacement.

• Start with 10 balls, 5 red and 5 black.

• Draw Ball 1. For that ball, \(P(red) = \frac{1}{2}\) and \(P(black) = \frac{1}{2}\).

• Draw Ball 2. For that ball, \(P(red)\) and \(P(black)\) will depend on color of Ball 1.

• Sampling without replacement = dependence

X-Minute Activity

• Question - “We start with an urn filled with 5 red and 5 black balls, randomly mixed. I draw three balls from the urn, sampling without replacement. What is the probability that at least one is black?”

• Let \(A\) = “Draw at least 1 black”, then \(A^C\) = “Draw 0 black”.

• We know \(P(A) + P(A^C) = 1\) and \(A^C\) has only one outcome: red, red, red.

• Final Answer: \[P(red,red,red) = \left(\frac{5}{10}\right) \left(\frac{4}{9}\right) \left(\frac{3}{8}\right) = \frac{1}{12}\] \[P(A) = 1 - P(A^C) = 1 – \frac{1}{12} = \frac{11}{12}\]

Summary

Today, we discussed the following:

• Independence vs Dependence

• Probability Urns Example

Next, we will discuss:

• More on Conditional probabilities

• Bayes’ theorem introduction