 
---
title: "Lab 8: Hypothesis Testing Using Infer"
author: "Insert Name"
output:
  pdf_document
 
---

## Instructions

**Due: Before lab Thursday or Friday April 1 or 2**



# Instructions
Work through the problems below (it may be helpful to have both the .rmd and .pdf files open side-by-side), and submit this document as a knitted .pdf file with answers to the Problem Set 7 assignment on [Gradescope](https://www.gradescope.com/courses/227442/). 


```{r, include=FALSE}
# Do not edit this code block/chunk
knitr::opts_chunk$set(echo = TRUE, message=FALSE, warning = FALSE, fig.width = 5, fig.height = 3)
```

### Load packages

For this lab we will read be using the `tidyverse` for data wrangling
and visualization and `oilabs` for the data set. `infer` is included as well, in case you decide to use it.

```{r, include=FALSE, message = FALSE, warning = FALSE}
library(tidyverse)
library(knitr)
library(oilabs)
library(infer)

set.seed(271)
```

 
# Background
  
As the novel coronavirus spread throughout 2020, the benefit of facemasks and other protective gear for preventing the transmission of airborne viruses received renewed attention. Historically, two types of disposable masks see frequent use: cloth/cotton surgical masks and heavier N95 respirators. In this lab, we will use hypothesis testing to answer the following question: is there a difference in effectiveness between the two types of masks?

## Data

C. MacIntyre et al compare the efficacy of cloth masks to medical masks in hospital healthcare workers in <https://bmjopen.bmj.com/content/5/4/e006577.long>

A total of 1607 healthcare workers were recruited in the study from 14 hospitals in Hanoi, Vietnam. Wards in participating hospitals were randomly sorted into 3 groups: a group instructed to wear `medical` masks at all times on shift, a group instructed to wear `cloth` masks at all times on shift, and a `control` group instructed to follow their standard routine for mask usage. Participants were followed for 4 weeks of facemask use and for one week after to determine the whether each exhibited symptoms of a clinical respiratory illness (`CRI`).


A subset of data from the study is presented in the following `masks` data frame.

```{r}
masks<-tibble(
  infected = c(
    replicate(n = 28,"yes" ),
    replicate(n = 552,"no" ), 
    replicate(n = 43,"yes" ), 
    replicate(n = 526,"no" ),
    replicate(n = 32,"yes" ), 
    replicate(n = 426,"no" )),
  mask_type = c(
    replicate(580, "medical"), 
    replicate(569, "cloth"), 
    replicate(458, "control"))
)
```

# Hypotheses and Data Exploration

One questions researchers are interested in resolving is whether the two masks were equally effective in preventing influenza-like illnesses. Without any additional information, the null hypothesis is that there is no difference between medical masks and cloth masks, and the alternative hypothesis is that there is a difference.

### Exercise 1

Using p_m and p_c to denote the theoretical infection rate among all health-care workers wearing **m**edical masks and **c**loth masks (respectively), write out the null and alternative hypotheses in mathematical notation.


********************************************

********************************************


### Exercise 2

 Use `ggplot` to create an appropriate data visualization that displays the proportion of health care workers that were infected in each group.


********************************************

********************************************


# Testing the hypothesis

We'll use `infer` to run this hypothesis test. 

## Step 1: Calculate the observed difference

While we could use `dplyr` verbs to calculate the difference in infection proportions between the `medical` and `cloth` masks groups, the `infer` package makes this calculation straightforward.


```{r}
masks_med_cloth<-masks %>% 
  filter(mask_type %in% c("medical", "cloth"))

obs_diff_masks <- masks_med_cloth %>%
  specify(infected ~ mask_type, success = "yes") %>% 
  calculate( stat = "diff in props", order = c("medical", "cloth"))
```

Notice in the above code that we needed to first restrict our attention only to those health care workers who wore cloth or medical masks (what group did we remove?)

We also needed to chose an order for the levels of the `mask_type` variable. The particular order we used doesn't matter, as long as we are consistent in all subsequent calculations.

### Exercise 3

What is the observed difference in proportion calculated above? Before you conduct any further analysis, make a guess: is this difference statistically significant?

********************************************

********************************************


## Step 2: Generate the permuted samples

This step involves generating simulated values of differences in proportions *as if* we lived in a world where there truly is no difference in effectiveness between the two mask types. Returning to the idea of permutation, this is akin to shuffling the `mask_type` label among all healthy care workers. 

```{r}
null_masks_samples <- masks_med_cloth %>% 
  specify(infected ~ mask_type, success = "yes") %>% 
  hypothesize(null = "independence") %>% 
  generate( reps = 1000, type = "permute")
```

### Exercise 4

What was the size of each shuffled sample? How many times did we shuffle? How many rows are there in the `null_masks_samples` data frame?

********************************************

********************************************

## Step 3: Create a null distribution of differences in proportions

The following code calculates the differences in infection rate between `medical` and `cloth` mask health-care workers.

```{r}
null_masks_proportions <- null_masks_samples %>% 
  calculate( stat = "diff in props", order = c("medical", "cloth"))
```

### Exercise 5

How many rows are there in the `null_masks_proportions` data frame? Why does this differ from the number of rows in the `null_masks_samples` data frame?

********************************************

********************************************

## Step 4: Visualize how the observed difference compares to the null distribution.

The following code plots the differences in proportions for each of the shuffled replicates.

```{r}
null_masks_proportions %>% 
  visualize()
```

**Note that zero is the center of this null distribution.** The null hypothesis is that there is no difference in infection rates between mask types. In the null distribution, zero was the most common difference, because we broke any existing relationship in the sample---infected healthcare workers were sorted into `medical` and `cloth` groups at random.

### Exercise 6

Based on this graphic, what do you think is a plausible range of values for the difference in proportions under the null hypothesis?

********************************************

********************************************


We can also visualize where the observed difference lies in the null distribution using the following code:

```{r warning = F}
null_masks_proportions %>% 
  visualize( ) +
  shade_p_value(obs_stat = obs_diff_masks, direction = "both")
```

**Note that since we used a two-sided alternate hypothesis, we set `direction = "both"` to indicate we should look for extreme values in both tails of the distribution. **

### Exercise 7

Based on this graphic, does the observed difference seem plausible if the null hypothesis were true? Explain.

********************************************

********************************************


## Step 5: Calculate a p-value

Use the followign code to calculate the p-value for the observed difference:

```{r}
null_masks_proportions %>% 
  get_p_value(obs_stat = obs_diff_masks, direction = "both")
```

The p-value indicates the probability that we would see a difference in proportion equal to, or more extreme, as the one we did if there were truly no difference in effectiveness between mask types.


### Exercise 8

Compare the p-value above to the standard signficance level of alpha = 0.05. Explain why this test does not give enough evidence to reject the null hypothesis. Does this imply that the null hypothesis is true?

********************************************

********************************************

## Exercise 9

The researchers in <https://bmjopen.bmj.com/content/5/4/e006577.long> actually drew the opposite conclusion, arguing that the data indeed showed a statistically significant difference in effectiveness between cloth and medical masks. This is because the researchers performed further statistical analysis to correct for the fact that the sample was generated using **cluster sampling** rather than **simple random sampling** (i.e. each health worker in the same ward was required to wear the same type of mask).

Theorize two reasons why the researchers may have decided to use cluster sampling rather than simple random sampling.

********************************************

********************************************

### Exercise 10

The 2010 General Social Survey asked 1,259 US residents:

>Do you think the use of marijuana should be made legal, or not?

48% of respondents said it should be made legal.

Run the following code to generate the sample for this `survey`.

```{r}
survey <- tibble(legalize = c(replicate(604, "yes"), replicate(655, "no") ) )
```


a. Is 48% a statistic or a parameter. Explain.

*********************************************************************************

*********************************************************************************

b. Does this sample give strong evidence that the proportion of support for the legalization of marijuana is NOT 50%? Justify your answer with a hypothesis test (Be sure to include your hypotheses and all code, in addition to your conclusion. )

*********************************************************************************

*********************************************************************************

c. Construct a 95% confidence interval for the true proportion of US residents who think marijuana should be made legal.

*********************************************************************************

*********************************************************************************

d. A news article on this survey states: "Majority of Americans think marijuana think marijuana should not be legalized." Based on your confidence interval, is this claim justified?

*********************************************************************************

*********************************************************************************


### Exercise 11

In March 2020, biomedical researchers in France conducted a study to test whether a popular anti-malaria drug was effective in reducing symptoms of the novel coronavirus. The report can be found here:

<https://www.sciencedirect.com/science/article/pii/S0924857920300996>

The results of the study were widely disseminated immediately after the report was released, and soon several world leaders suggested that a promising cure for coronavirus had been identified. However, many in the biomedical statistics community expressed skepticism about the results. 

In this problem, we will first perform a hypothesis testing imitating the procedure used by the study's authors, and then will investigate some methodological flaws in the original study.

The original study separated patients testing positive for coronavirus into two groups: a treatment group consisting of 20 patients receiving a daily doses of hydroxychloroquine sulfate 200mg, and a control group consisting of 16 patients receiving ``standard'' coronavirus care. Patients in each group were tracked for 6 days, and the number of patients in each group no longer test positive for the virus on the 6th day were recorded.

Run the following code to load the results in the `coronavirus` data frame.

```{r}
coronavirus <- tibble(
  test_corona = c(replicate(6, "positive"), replicate(14, "negative") , replicate(14, "positive"), replicate(2, "negative") ),
  group = c(replicate(20, "treatment"), replicate(16, "control"))
                      )
```


a. State the researchers null and alternative hypotheses, both in words and mathematical symbols.

*********************************************************************************

*********************************************************************************

b. Perform a hypothesis test at the alpha = 0.01 significance level.

*********************************************************************************

*********************************************************************************

c. Based on the P-value obtained in the test, make a conclusion about the researcher's hypotheses.

*********************************************************************************

*********************************************************************************

d. In the experiment design, the researchers elected to use as a control the patients who refused treatment or who had an exclusion criterion (i.e could not safely be given the treatment medication due to pre-existing conditions like high blood pressure). Explain why should cause skepticism about your conclusion in part (c). Does the size of the P-value in computed in the previous part overcome this skepticism?

*********************************************************************************

*********************************************************************************

e. Read the following March 30, 2020 article published in the *Annals of Internal Medicine*:

<https://www.acpjournals.org/doi/10.7326/M20-1223>

What are two other methodological flaws the authors of this article identified in the original study?

*********************************************************************************

*********************************************************************************


### Exercise 12

The consumption of caffeine to benefit alertness is a common activity practiced by 90% of adults in North America. Often, caffeine is used in order to replace the need for sleep. One recent study compares student's ability to recall memorized information after either the consumption of caffeine or a brief nap. A random sample of 35 adults were randomly divided into 3 groups and verbally given a list of 24 words to memorize. During a break, one of the groups was instructed to take a nap for an hour and a half, another group is kept awake and given a caffeine pill an hour prior to testing, and the third group is kept awake and given a placebo. The response variable of interest is the number of words participants are able to recall following the break.

Summary statistics for the three groups are presented in the table below. We are interested in testing whether there is evidence of a difference in average recall ability between any two of the treatments. Thus, we have 3 possible tests: Sleep vs. Caffeine, Sleep vs. Placebo, Caffeine vs Placebo.

| **Group**  | **Sample Size**   | **Mean** | **St. Dev**  | 
|------------|-------------------|----------|--------------| 
| Sleep    | 12 | 15.25  | 3.3  | 
| Caffeine | 12 | 12.25  | 3.5  | 
| Placebo  | 11 | 13.70  | 3.0  | 


a. In the test comparing the sleep to caffeine group, the p-value is 0.003. What is the conclusion of the test? In the sample, which group had the better recall ability? According to the hypothesis test, can you conclude that sleep is actually better than caffeine for recall ability (carefully consider the implied alternate hypothesis in the problem statemen)?

*********************************************************************************

*********************************************************************************

b. In the test comparing the sleep group to the placebo group, the p-value is 0.06. What is the conclusion of the test using a 5% significance level? Using a 10% significance level? How strong is the evidene of a difference in mean recall ability betwen these two groups?

*********************************************************************************

*********************************************************************************

c. In the test comparing the caffeine group to the placebo group, the p-value is 0.22. What is the conclusion of the test? In the sample, which group had better recall ability? According to the hypothesis test, are we justified in concluding that caffeine consumption has no effect on recall ability?

*********************************************************************************

*********************************************************************************

d. Just based on the results of these samples alone, what seems to be the best strategy to implement before a memory-intensive exam? Explain why this doesn't mean the result you chose has the highest likelihood of increasing your exam score.

*********************************************************************************

*********************************************************************************

### Exercise 13

For each of the following situations, describe the effects of a Type I and Type II error in context. Then indicate whether it makes more sense to use a larger significance level (such as alpha = 0.10) or a smaller significance level (such as alpha = 0.01).

a. A new drug with potentially dangerous side effects is tested to see if it is better than the drug currently in use. If it is found to be effective, it will be prescribed to millions of people.

*********************************************************************************

*********************************************************************************

b. Two slightly different versions of a statistics exercise are given to the 10am and 11am sections of Math 141 and the proportions of students in each class who get the question correct are recorded. We wish to determine whether one question was more difficult than the other.

*********************************************************************************

*********************************************************************************

c. A consumer advocacy group wishes to test if a popular company is engaging in false advertising. If there is evidence the company is lying, the Federal Trade Commission will file a lawsuit against the company.

*********************************************************************************

*********************************************************************************

d. You test whether taking a vitamin supplement reduces the duration of symptoms of the common cold. There are no (known) harmful side effects of the supplement.

*********************************************************************************

*********************************************************************************

e. A government intelligence agency notices increased activity in online chatrooms often frequented by members of terrorist group. If they forward the report to the executive branch, a pre-emptive military strike will be ordered against a suspected terrorist compound in a foreign country.

*********************************************************************************

*********************************************************************************