Previously on Probability versus Statistics…

Previously on Statistical terms…

Statistical terms we learned and how it applies to problems:

1. Population - everyone or everything of interest in the context of a research question.

2. Sample - a fraction of the population taken either randomly or selectively.

3. Sampling Principles and Strategies: (1) Simple random - each item is equally likely to get sampled, (2) Stratified - items are divided into strata and then a second sampling method, usually simple random sampling, is employed within each stratum, (3) Clustered - items are placed into many groups, called clusters and then we sample a fixed number of clusters and include all observations from each of those clusters in the sample.

4. Types of studies: (1) Observational - samples are observed or certain outcomes are measured without trying to change, control, or affect them, (2) Experimental - a method where researchers introduce an intervention or control to study its outcomes - usually unknown.

Previously on Statistical terms…

Statistical investigation:

1. Collect/produce data: the data we get is simply the sample of a target population.

2. Data exploration: summary statistics visualization, regression, examining relationships, etc…

3. Analysis and Conclusions: use the data and statistical inference to draw a conclusion about the target population.

How do we know that the observations/data we have is unlikely to have occurred by chance?

What do we mean by “unlikely to have occurred by chance”?

Let’s learn about probability first before we can proceed into hypothesis testing.

Introduction to Sets and Basic Probability

In this lecture, we will learn about

• Sample space and events.

• Basic probability rules.

• Set notations on probabilities.

What is probability?

• Probability is the branch of mathematics that deals with randomness.

• The likelihood of an event happening.

• An extent to which an event is likely to occur which is \[probability = \frac{an \hspace{10px} event}{all \hspace{10px} possible \hspace{10px} events}.\]

Is the coin fair?

Say we found a coin somewhere and we ask…

Do we have a fair coin?

• Suppose you want to flip the coin and see what comes up.

• How can you tell if a coin is fair? Here, “Fair” means equal chance of getting a head (H) or tail (T).

• First, let’s learn some terminologies and determine what a fair coin should be…

Sample Space, and Events

• The sample space is the set of all possible outcomes.
  Coin Example: \(\Omega = \{H,T\}\)

• An event is any subset of the sample space. An event space contains all subsets of outcomes of the sample space.
  Coin example: \(\{H\}\), \(\{T\}\), \(\{H,T\}\), \(\emptyset\)
  \(\{H\}\) - the coin lands on Heads.
  \(\{T\}\) - the coin lands on Tails.
  \(\{H,T\}\) - the coin lands on Heads OR Tails.
  \(\emptyset = \{\}\) - the empty set

• A disjoint event is an event where the intersection of two sets are empty.
  Coin Example:
  \(\{H\}\) and \(\{T\}\) are disjoint.
  \(\{H,T\}\) and \(\{T\}\) are NOT disjoint.

Set Notation

Suppose we have events A and B:

• \(\cap\) means “intersection” “\(A \cap B\)” is the set of all objects in A AND B

• \(\cup\) means “union” “\(A \cup B\)” is the set of all objects in A OR B.

• Disjoint events: A and B are disjoint if \(A \cap B = \emptyset\)

Fair Coin

What is a fair coin? “Fair” means equal chance of getting a head (H) or tail (T).

• For the fair coin, we have
  \(P(H) = \frac{1}{2} \longrightarrow\) the probability of heads is one half.
  \(P(T) = \frac{1}{2} \longrightarrow\) the probability of tails is one half.

• The symbol \(P\) is known to be the probability function which maps the events in a sample space to a value between 0 and 1.

• Applying the set notation:
  \(P(\{H,T\}) = 1\)
  because \(\{H,T\}\) is the union of disjoint events \(\{H\}\) and \(\{T\}\).
  Specifically, we have \(P(\{H,T\}) = P(\{H\}) + P(\{T\}) = \frac{1}{2} + \frac{1}{2} = 1\)

Fair Coin

A fair coin has the following idealized scenario:

• \(P(\{H\}) = \frac{1}{2}\)

• \(P(\{T\}) = \frac{1}{2}\)

• \(P(\{H,T\}) = 1\)

• \(P(\emptyset) = 0\)

Laws of Probability Functions

Probability functions must satisfy the following basic rules:

• The sum of the probabilities for all outcomes in the sample space is equal to 1: \[P(\Omega) = 1\]
• The empty set has 0 probability: \[P(\emptyset) = 0\]
• Probabilities are always positive (never negative) and always between 0 and 1. \[P \in [0,1]\]
• If events A and B are disjoint (\(A \cap B = \emptyset\)), then \[P(A \cup B) = P(A) + P(B)\]

Fair Coin

We know that for a fair coin, \(P({H}) = \frac{1}{2}\) and \(P({T}) = \frac{1}{2}\).

Now, let’s do an experiment…

• Let’s say we flip a fair coin three times.

• Question: How many possible outcomes are there?

• Answer: Since there are 2 outcomes per flip, then the number of possible outcomes is \[2 \times 2 \times 2 = 8.\]

• The sample space for three coin flips is written as \[\Omega = \{\{H,H,H\},\{H,H,T\},\{H,T,T\},\{T,T,T\},\{T,T,H\},\{T,H,H\},\{H,T,H\},\{T,H,T\}\}\]

Inpependence

What is the probability function for our fair coin? We need to determine the probability for each outcome.

• For a fair coin, \(P({H}) = \frac{1}{2}\) and \(P({T}) = \frac{1}{2}\) which does not change as function of when we flip the coin.

• We call this independent events.

• Consequently, for independent events A and B, \(P(A \cap B) = P(A)P(B)\).

• Question: What is the probability of getting \(\{H,H,H\}\).

• Answer: \(P(\{H,H,H\}) = P(H)P(H)P(H) = \frac{1}{2} \frac{1}{2} \frac{1}{2} = \frac{1}{8}\)

• In fact, the probability of getting any outcome from the sample space is \(\frac{1}{8}\) if we consider that order matters.

Independence

Recall that the sample space for three coin flips is \[\Omega = \{\{H,H,H\},\{H,H,T\},\{H,T,T\},\{T,T,T\},\{T,T,H\},\{T,H,H\},\{H,T,H\},\{T,H,T\}\}\]

• Question: If we flip the coin three times, what is the probability of getting exactly two heads in any order?

• Answer: \(P(\{\{H,H,T\},\{T,H,H\},\{H,T,H\}\}) = \frac{3}{8}\)

• Question: If we flip the coin three times, what is the probability of getting at least two heads in any order?

• Answer: \(P(\{\{H,H,H\},\{H,H,T\},\{T,H,H\},\{H,T,H\}\}) = \frac{4}{8} = \frac{1}{2}\)

• Question: If we flip the coin three times, what is the probability of getting at most two heads in any order?

• Answer: \(P(\{T,T,T\},\{\{H,T,T\},\{T,T,H\},\{T,H,T\},\{H,H,T\},\{T,H,H\},\{H,T,H\}\}) = \frac{7}{8}\)

Independence

Let \(x_i\) be the outcome of the \(i^{th}\) flip.

We can write an outcome of three flips as \[(x_1,x_2,x_3)\] Then, we define the probability function for three coin flips as \[P(x_1,x_2,x_3) = P(x_1)P(x_2)P(x_3) \]

Is the coin fair?

• We know that for a fair coin, \(P({H}) = \frac{1}{2}\) and \(P({T}) = \frac{1}{2}\).

• If I flip this coin multiple times, I should expect roughly 50% heads and 50% tails for a fair coin while also accounting for variation in my samples. Ideally, we want a lot of samples.

• Suppose that I flip the coin 1000 times and we get 504 heads and 496 tails. Likely to have occurred by chance because we know there is going to be some variation when we flip the coin a finite number of times and we expect it to be close to 50% heads and 50% tails.

• Suppose that I flip the coin 1000 times and we get 200 heads and 800 tails. Unlikely to have occurred by chance because it does not seem likely if the coin is truly fair. Then, you would think that this coin is sus.
  How can we prove statistically that the coin is unfair? We will talk about the foundation of inference, hypothesis testing and confidence intervals in the next two weeks.

Summary

In this lecture, we talked about the following:

• The Sample space

• Events and independent events

• Joint (not disjoint) and Disjoint

• Basic probability rules

In the next lecture, we will talk about,

• Discrete and Continuous Random variables

• More on probability functions

• The law of large numbers

Today’s Activity

Work within your groups to discuss your answers for the following questions.

Four coin flips. Suppose we flip a coin four times.

1. How many possible outcomes are there?

2. What is the probability of getting four tails?

3. What is the probability of getting exactly three tails?

4. What is the probability of getting at most three tails?

5. What is the probability of getting at least three tails?