Previously on Probability…

Last session, we discussed about:

• The Sample space

• Events and independent events

• Joint (not disjoint) and Disjoint

• Basic probability rules

Basic Probability Continued

In this lecture, we will learn about,

• Discrete and Continuous Random variables

• More on probability functions

• A light introduction to the law of large numbers (chapter 13 in your textbook)

Random Variable

A Random Variable (R.V.) is a type of variable where the value is a function that associates a numerical value with a potential outcome.

• Random Variables are denoted by upper case letters (\(Y\))

• Individual outcomes for R.V. are denoted by lower case letters (\(y\))

• Having R.V.s in mathematical notations is helpful for writing complicated probabilities.

Tossing a Fair Coin ONE time

• Sample space:

  \[\Omega = \{H,T\}\]

• Let \(Y\) be random variable representing flipping a coin, and \(y\) be the outcome of landing head (\(y=1\)) or tail (\(y=0\));

• Here, \(\Omega\) is a discrete sample space and \(X\) is a discrete R.V.

Tossing a Fair Coin THREE Times

• Sample space: \[\Omega = \{\{H,H,H\},\{H,H,T\},\{H,T,T\},\{T,T,T\}, \\ \{T,T,H\},\{T,H,H\},\{H,T,H\},\{T,H,T\}\}\]

• Let the random variable \(X\) to be the number of heads in this experiment, \[X = \{x_1,x_2,x_3\} = {0,1,2,3}\]

• Example probabilities:
  

  • \(P(X = 3) = \frac{1}{8} \longrightarrow\) the probability of getting exactly three heads
    
  • \(P(X = 2) = \frac{3}{8} \longrightarrow\) the probability of getting exactly two heads
    
  • \(P(X \ge 2) = \frac{1}{2} \longrightarrow\) the probability of getting at least two heads
    
  • \(P(X \le 2) = \frac{7}{8} \longrightarrow\) the probability of getting at most two heads
    

• Here, \(\Omega\) is a discrete sample space and \(X\) is a discrete R.V.

Discrete R.V. vs Continuous R.V.

• Discrete Random Variable: An R.V. that can take on only a finite or countably infinite set of outcomes. We can say that a discrete R.V. has distinct values that can be counted.
  
  Example: Coin Flips, Dice Rolls, Outcomes of a Test (positive/negative), gender (M/F), etc.

• Continuous Random Variable: An R.V. that can take on any value along a continuum (but may be reported “discretely”)
  
  Example: Height (really continuous, but we usually just report to the nearest inch/centimeter), temperatures, etc.

• Probability Functions = Probability Distributions Table, Graph, or Formula that describes values a random variable can take on, and its corresponding probability (discrete R.V.) or density (continuous R.V.).

Discrete R.V. vs Continuous R.V.

Tossing a Fair Coin Once

• For one coin toss, we know the PMF to be \[P(Y = k) = \begin{cases} p & \text{if } k=1 \text{ (H)} \\ 1-p & \text{if } k=0 \text{ (T)}\end{cases}\] where \(p=\frac{1}{2}\) and \(Y\) is the discrete R.V. for the outcome of the coin toss to be heads or not heads (which is tails).

• This function is also known as the Bernoulli Distribution, a statistical model for only two possible outcomes (yes/no,success/failure,etc.) of any single experiment.

• We can rewrite the above function as \[P(Y=k) = p^k (1-p)^{1-k}\]

Tossing a Fair Coin \(\mathbf{n}\) Times

• Previously, we determined the state space of tossing a coin three times. The sample space of tossing a fair coin three times is \(2^3 = 8\). That was easy to list all of the possible outcome.

• It gets complicated if we increase the number of tosses.

• For example:
  \(2^2 = 2 \longrightarrow\) two tosses
  \(2^3 = 8 \longrightarrow\) three tosses
  \(2^4 = 16 \longrightarrow\) four tosses
  \(2^5 = 32 \longrightarrow\) five tosses
  \(\vdots\)
  \(2^{20} = 1048576 \longrightarrow\) twenty tosses
  \(\vdots\)
  \(2^{200} = 1.606938 \times 10^{60} \longrightarrow\) two hundred tosses
  

• The number of total possible outcomes in the sample space increases by powers of two as \(n\) increases.

Tossing a Fair Coin \(\mathbf{n}\) Times

• Now, let’s simplify this example by defining an R.V.

• Let the discrete R.V. \(X\) to be the number of heads (say observing heads is considered as a success). We want to define a probability mass function (PMF) to compute the probability of observing the number of heads (observing the number of successes).

• Let \(n\) be the number of tosses and \(p\) be the probability of success. In this example, \(p=\frac{1}{2}\).

• The PMF that models multiple coin tossing is the Binomial Distribution, which is used when there are exactly two disjoint (also called mutually exclusive) outcomes of a trial (in this case, a coin toss).

• These outcomes are labeled “success” (observing heads) and “failure” (observing not heads - which tails).

Bernoulli’s Beau

• The general form of the Binomial Distribution is given by \[P(X = x; n, p) = {n \choose x} p^x (1-p)^{(n-x)} \hspace{5px} \text{ for } x = 0,1,2,\cdots, n\] where \({n \choose x} = \frac{n!}{x!(n-x)!}\).

• The binomial distribution assumes that \(p\) is fixed for all \(n\) trials.

• The expected value or the mean of the binomial PMF is \(E[X] = np\). For example, if \(n=20\) and \(p=\frac{1}{2}\), \(E[X] = 10\).

• Notice that the term \(p^x (1-p)^{(n-x)}\) is the Bernoulli. Here, a trial is also called a Bernoulli trial, a random experiment with exactly two possible outcomes. Bernoulli is Binomial’s bae, basically.

• Don’t worry about the probability function and let’s accept it as it is … for now.

• In R, you can use the dbinom function to compute the same probability distribution.

Tossing a Fair Coin \(\mathbf{n = 20}\) Times

Tossing a Fair Coin \(\mathbf{n = 20}\) Times

Tossing a Fair Coin \(\mathbf{n = 20}\) Times

Tossing a Fair Coin \(\mathbf{n = 20}\) Times

Tossing Coins Simulations

• We are uncertain on predicting the outcome of one coin toss but we have an expected value that with - for example - 20 tosses, we will get about 10 heads, which is half of 20.

• Suppose that we simulate tossing coins with increasing number of tosses (trials) and record the cumulative number heads.

• For smaller number of tosses, we will see a lot of variation but as the number of tosses increases, the proportion of heads gets closer to \(\frac{1}{2}\) or the number of heads gets closer to the expected value.

Tossing Coins Simulations

The Law of Large Numbers

Summary

In this lecture, we talked about the following:

• Discrete and continuous random variables

• The Bernoulli and the Binomial probability mass functions (pmf)

• A light introduction to the law of large numbers

In the next lecture, we will talk about,

• More on continuous random variables

• Conditional Probabilities

• A light introduction to the central limit theorem (chapter 13 in your textbook)

• Prelude to hypothesis testing and confidence intervals (chapter 11 and 12 in your textbook)

Today’s Activity

Work within your groups to discuss the answers of the following problems.

Part A: Consider rolling a six-sided dice two times.

1. How many possible events are there? Make a table of possible events of rolling a two-sided dice twice and make another table that sums the numbers of each possible event.

2. What is the probability of rolling a (3,4) pair - in order?

3. What is the probability of rolling a sum of 8 or a sum of 6?

Part B: Consider binomial experiment where observing a six is a success (not observing a six is a failure). Suppose that the number of rolls is 3.

1. What is the probability of observing a six at least 2 times? You can use the dbinom function in R.

Part C: The Binomial distribution simulates the number of successes in \(n\) independent trials, each of which is a “success” with probability \(p\) and a “failure” with probability \(1-p\). Discuss real-life applications of the binomial experiment.