Previously on Bayes’ Theorem…

We used Bayes’ Theorem to do some examples
- Balls in Jars Example as the simplied example to used Bayes’ Theorem
- Defined accuracy, precision, sensitivity, and specificity for a binary contingency table.

COVID Testing Accuracy

Example: There is no test that is 100% accurate in detecting COVID. Suppose that there is a 95% percent accuracy in detecting COVID infections.

What does accuracy mean?

Useful terms:

True positive: Person test positive with the actual COVID.
False positive: Person tests positive without the actual COVID.
False negative: Person tests negative with the actual COVID.
True negative: Person tests negative without the actual COVID.

Accuracy vs Precision

	positive test \(+\)	negative test \(-\)
COVID \(C^+\)	true positives (tp)	false negative (fn) type II error	true positive rate (sensitivity) \(\frac{tp}{tp+fn}\)
No COVID \(C^-\)	false positive (fp) type I error	true negative (tn)	true negative rate (specificity) \(\frac{tn}{tn+fp}\)
	positive predicted value (precision) \(\frac{tp}{(tp+fp)}\)	negative predicted value \(\frac{tn}{tn+fn}\)	accuracy \(\frac{tp+tn}{tp+tn+fp+fn}\)

100,000 Patients Example

Suppose that we have \(100,000\) patients with \(1000\) patients are infected and \(99,000\) patients are not, and below shows how many of them tested positive and tested negative. Given this example, the probability of having actual COVID is 1% (which is also known as the prevalence rate).

	positive test \(+\)	negative test \(-\)
COVID \(C^+\)	950	50	\(\text{sensitivity} = \frac{950}{1000} = 0.95\)
No COVID \(C^-\)	4950	94050	\(\text{specificity} = \frac{94050}{99000} = 0.95\)
	\(\text{precision} = \frac{950}{950+4950} = 0.1610\)		\(\text{accuracy} = \frac{950+94050}{100,000} = \mathbf{0.95}\)

Sensitivity and Specificity

Sensitivity: the probability of a person tests positive with the actual COVID (this is the true positive rate) \[P(+|C^+) = 0.95\]
Specificity: the probability of a person tests negative without the actual COVID (this is the true negative rate) \[P(-|C^-) = 0.95\]

How is this information useful? Suppose we randomly select a person from a population and they tested positive. What is the probability that they have COVID? Here, we want to compute the conditional probability \[P(C^+|+).\]

Applying Bayes’ Theorem (1/2)

The Bayes’ theorem expression for our example is \[P(C^+|+) = \frac{P(C^+)P(+|C^+)}{P(+)}\] where \(P(+)\) is computed using the law of total probability written as \[P(+) = P(C^+)P(+|C^+) + P(C^-)P(+|C^-).\]

\[P(+|C^-) = 1 - P(-|C^-) = 1 - 0.95 = 0.05 \longrightarrow \text{false positive rate}\]

\[P(+|C^+) = 0.95 \longrightarrow \text{true positive rate}\]

Suppose that there is a 1% COVID cases in the area where we sampled the person. \[P(C^+) = 0.01 \longrightarrow \text{prior probability (or prevalence rate in epidemiology)}\]

Applying Bayes’ Theorem (2/2)

Then, we can compute the marginalization probability: \[ \begin{align} P(+) & = P(C^+)P(+|C^+) + P(C^-)P(+|C^-) \\ & = (0.01)(0.95) + (1-0.01)(0.05) = 0.059 \end{align} \]

Therefore, \[P(C^+|+) = \frac{P(C^+)P(+|C^+)}{P(+)} = \frac{(0.01)(0.95)}{0.059} = 0.1610\]

Interpretation of \(P(C^+|+) = 0.1610\): There is approximately 16.10% chance that a randomly sampled person - from an area with known 1% of the population who have COVID - having actual COVID given that they test positive.

15.15-Minute activity (1/4)

Suppose that you randomly sample another person

A location has 0.05% COVID cases (prevalence). Given that they tested positive, what is the probability that they have COVID? Interpret this probability.
Try 3% COVID cases (prevalence). Did the probability increase or decrease from problem 1? Interpret this probability and compare this to the result from problem 1.

Information:

\[P(+|C^-) = 1 - P(-|C^-) = 1 - 0.95 = 0.05 \longrightarrow \text{false positive rate}\]
\[P(+|C^+) = 0.95 \longrightarrow \text{true positive rate}\]

Timer starts

10:10

15.15-Minute activity (2/4)

Answer: Givens \(P(C^+) = 0.0005\), \(P(+|C^-) =0.05\), and \(P(+|C^+) = 0.95\). \[ \begin{align} P(+) & = P(C^+)P(+|C^+) + P(C^-)P(+|C^-) \\ & = (0.0005)(0.95) + (1-0.0005)(0.05) = 0.0505 \end{align} \] \[P(C^+|+) = \frac{P(C^+)P(+|C^+)}{P(+)} = \frac{(0.0005)(0.95)}{0.0505} = 0.0094\]

There is a 0.94% chance that a randomly selected person have COVID given that they testing positive.

15.15-Minute activity (3/4)

Answer: Givens \(P(C^+) = 0.03\), \(P(+|C^-) =0.05\), and \(P(+|C^+) = 0.95\). \[ \begin{align} P(+) & = P(C^+)P(+|C^+) + P(C^-)P(+|C^-) \\ & = (0.03)(0.95) + (1-0.03)(0.05) = 0.077 \end{align} \] \[P(C^+|+) = \frac{P(C^+)P(+|C^+)}{P(+)} = \frac{(0.03)(0.95)}{0.077} = 0.3701\]

There is a 37.01% probability that a randomly selected person have COVID given that they testing positive. This result is way higher than the previous but still less than 50% chance.

What does this all mean? (1/2)

Since we randomly sampled a person from a general population with prevalence rate 0.05% to get tested, the probability of having actual COVID given that they test positive is quite low. This is due to the sampling procedure rather than the accuracy of the test. This is a problem on rare diseases.
For a prevalence rate of 3%, for a randomly sampled person, the probability of having actual COVID given that they test positive is higher - around 37% chance. This tells us that the true positive and true negative rates, and the prevalence rate including specific information on when and where the person was sampled plays a key role on determining the chances of having actual COVID given that the person tested positive.
Despite the fact that the test produce reliable findings, because the probability of someone having COVID is so low (prevalence rate is less than 5%), false positives account for the vast majority of positive test results.

Summary

Today, we discussed the following:

Defined accuracy, precision, sensitivity, and specificity on a binary contingency table.
COVID testing examples.

Next, we will discuss:

Inference for single proportion

10 - Bayes’ Theorem Plus