 
---
title: "Lab 7: Confidence Intervals"
author: "Insert Name"
output:
  pdf_document
 
---

# Instructions


**Due: Before lab Thursday or Friday March 25 or 26**

# Instructions
Work through the problems below (it may be helpful to have both the .rmd and .pdf files open side-by-side), and submit this document as a knitted .pdf file with answers to the Problem Set 7 assignment on [Gradescope](https://www.gradescope.com/courses/227442/). 




```{r, include=FALSE}
# Do not edit this code block/chunk
knitr::opts_chunk$set(echo = TRUE, message=FALSE, warning = FALSE, fig.width = 5, fig.height = 3)
```


## Background

If you have access to data on an entire population, say the opinion of every adult in the United States on whether or not they think climate change is affecting their local community, it's straightforward to answer questions like, "What percent of US adults think climate change is affecting their local community?". 

Similarly, if you had demographic information on the population you could examine how, if at all, this opinion varies among young and old adults and adults with different leanings.

But if you only have access to a sample of the population, as is often the case, the task becomes more complicated. What is your best guess for this proportion if you only have data from a small sample of adults? This type of situation requires that you use your sample to make inference on what your population looks like.

## An Important Note on Reproducibility

You will take random samples and build sampling distributions in this lab, which means you should set a seed using `set.seed()` in each code chunk where you generate random values to ensure that each time you knit your document, you obtain an identical random sample.


### Load packages

In this lab, we will explore and visualize the data using the **tidyverse** suite of packages, and perform statistical inference using **infer**.

Let's load the packages.

```{r load-packages, message=FALSE}
library(tidyverse)
library(openintro)
library(moderndive)
library(infer)
``` 
 
### The data


To keep our computation simple, we will assume a total population size of 100,000 (even though that's smaller than the population size of all US adults).

A 2019 Pew Research report states the following:

> Roughly six-in-ten U.S. adults (62%) say climate change is currently affecting their local community either a great deal or some, according to a new Pew Research Center survey.
>
>**Source:** [Most Americans say climate change impacts their community, but effects vary by region](https://www.pewresearch.org/fact-tank/2019/12/02/most-americans-say-climate-change-impacts-their-community-but-effects-vary-by-region/)
In this lab, you will assume this 62% is a true population proportion and learn about how sample proportions can vary from sample to sample by taking smaller samples from the population. 
We will first create our population assuming a population size of 100,000. 
This means 62,000 (62%) of the adult population think climate change impacts their community, and the remaining 38,000 does not think so.

```{r}
us_adults <- tibble(
  climate_change_affects = c(rep("Yes", 62000), rep("No", 38000))
)
```

The name of the data frame is `us_adults` and the name of the variable that contains responses to the question *"Do you think climate change is affecting your local community?"* is `climate_change_affects`.

We can quickly visualize the distribution of these responses using a bar plot.

```{r bar-plot-pop, fig.height=2.5, fig.width=10}
ggplot(us_adults, aes(x = climate_change_affects)) +
  geom_bar() +
  labs(
    x = "", y = "",
    title = "Do you think climate change is affecting your local community?"
  ) +
  coord_flip() 
```

We can also obtain summary statistics to confirm we constructed the data frame correctly.

```{r}
us_adults %>%
  count(climate_change_affects) %>%
  mutate(p = n /sum(n))
```

In this lab, you'll start with a simple random sample of size 60 from the population.

```{r sample}
set.seed(1)

samp <- us_adults %>%
  rep_sample_n(size = 60, reps = 1)
```


### Exercise 1
What percent of the adults in your sample think climate change affects their local community? **Hint:** Just like we did with the population, we can calculate the proportion of those **in this sample** who think climate change affects their local community. Be sure to change the value of the seed in the following chunk to a number of your choice.

****************************************************

****************************************************

### Exercise 2
Would you expect another student's sample proportion to be identical to yours? Would you expect it to be similar? Why or why not? Check with another student in the class to verify your hypotheses.

****************************************************

****************************************************

## Confidence intervals

Return for a moment to the question that first motivated this lab: based on this sample, what can you infer about the population? 
With just one sample, the best estimate of the proportion of US adults who think climate change affects their local community would be the sample proportion, usually denoted as $\hat{p}$ (here we are calling it `p_hat`). 
That serves as a good **point estimate**, but it would be useful to also communicate how uncertain you are of that estimate. 
This uncertainty can be quantified using a **confidence interval**.

One way of calculating a 95% confidence interval for a population proportion is based on the Central Limit Theorem, as $\hat{p} \pm 2 SE_{\hat{p}}$ is, or more precisely, as
\[ \hat{p} \pm 2 \sqrt{ \frac{\hat{p} (1-\hat{p})}{n} } \]

Another way is using simulation, or to be more specific, using **bootstrapping**. 
The term **bootstrapping** comes from the phrase "pulling oneself up by one’s bootstraps", which is a metaphor for accomplishing an impossible task without any outside help.
In this case the impossible task is estimating a population parameter (the unknown population proportion), and we’ll accomplish it using data from only the given sample.
Note that this notion of saying something about a population parameter using only information from an observed sample is the crux of statistical inference, it is not limited to bootstrapping. 

In essence, bootstrapping assumes that there are more of observations in the populations like the ones in the observed sample. 
So we "reconstruct" the population by resampling from our sample, with replacement. 
The bootstrapping scheme is as follows:

- **Step 1.** Take a bootstrap sample - a random sample taken **with replacement** from the original sample, of the same size as the original sample.
- **Step 2.** Calculate the bootstrap statistic - a statistic such as mean, median, proportion, slope, etc. computed on the bootstrap samples.
- **Step 3.** Repeat steps (1) and (2) many times to create a bootstrap distribution - a distribution of bootstrap statistics.
- **Step 4.** Calculate the bounds of the XX% confidence interval as the middle XX% of the bootstrap distribution.

Instead of coding up each of these steps, we will construct confidence intervals using the **infer** package.

Below is an overview of the functions we will use to construct this confidence interval:

Function     | Purpose
------------:|:------------------------------------------
`specify`    | Identify your variable of interest
`generate`   | The number of samples you want to generate
`calculate`  | The sample statistic you want to do inference with, or you can also think of this as the population parameter you want to do inference for
`get_ci`     | Find the confidence interval
`visualize`  | Create a histogram of the bootstrap statistics

This code will create a bootstrap distribution based on your sample. 

```{r}
set.seed(1)

bootstrap_dist<- samp %>%
  specify(response = climate_change_affects, success = "Yes") %>%
  generate(reps = 1000, type = "bootstrap") %>%
  calculate(stat = "prop")  
```

The code in the next chunk will compute the 95% confidence interval using the percentile method for the proportion of US adults who think climate change affects their local community.

```{r}
percentile_ci<-bootstrap_dist %>% 
  get_confidence_interval(level = 0.95, type = "percentile")
percentile_ci
```


We can also compute a confidence interval using the standard error method, **provided the bootstrap distribution appears symmetric, unimodal and bell-shaped.** To do so, we first need to store the sample statistic as a variable, as shown in the following code chunk. 

```{r}
sample_statistic<- samp %>%
  specify(response = climate_change_affects, success = "Yes") %>%
#  generate(reps = 1000, type = "bootstrap") %>%
  calculate(stat = "prop")  
sample_statistic
```
Note that we use the same code as we did for the bootstrap distribution, except we **removed** the `generate` verb by commenting out the line using `#` (equivalently, we could also have just deleted the line).

The next code chunk will compute the 95% confidence interval using the standard error method, which follows a similar pattern to the 95% interval using the percentile method, except we need to additionally specify the statistic we are using as our `point estimate`.

```{r}
se_ci<-bootstrap_dist %>% 
  get_confidence_interval(level = 0.95, type = "se", point_estimate = sample_statistic)
se_ci
```


The code in the following chunk will create a histogram of the bootstrap statitsics.

```{r}
bootstrap_dist %>% 
  visualise( )
```

We can add an additional layer to `visualize` in order to shade the region corresponding to the confidence interval, as demonstrated in the following code:

```{r}
bootstrap_dist %>% 
  visualise( )+ 
  shade_confidence_interval(endpoints = percentile_ci)
```



To summarize:
- In `specify` we specify the `response` variable and the level of that variable we are calling a `success`.
- In `generate` we provide the number of resamples we want from the population in the `reps` argument (this should be a reasonably large number) as well as the type of resampling we want to do, which is `"bootstrap"` in the case of constructing a confidence interval.
- Then, we `calculate` the sample statistic of interest for each of these resamples, which is `prop`ortion.
- Finally, we may either `visualize` the bootstrap distribution by creating a histogram of the numerical stat variable’s values, or `get_confidence_interval` and specify the type (either `"percentile"` or `"se"`) and confidence `level`.




## Confidence levels


### Exercise 3a

Follow the procedures outlined above to create a new sample called `samp50` of size 50. Create a bootstrap distribution called `bootstrap_dist50`, plot it, and calculate the 95% confidence interval using the percentile method.

**************************************************

**************************************************


### Exercise 3b

In this case, we have the rare luxury of knowing the true population proportion (62%) since we have data on the entire population. Does your confidence interval capture the true population proportion of US adults who think climate change affects their local community? If you are working on this lab in a classroom, does your neighbor's interval capture this value? 

**************************************************

**************************************************

### Exercise 3c

Each student should have gotten a slightly different confidence interval. What proportion of those intervals would you expect to capture the true population proportion? Why?

**************************************************


**************************************************


### Exercise 4

Since we know the population parameter, we can compute the sampling distribution. Plot a sampling distribution of 1000 samples taken from the population. Compare it to the plot of the bootstrap distribution you created in Exercise 3a. Do the distributions have the same center? Spread? Shape? 

**************************************************


**************************************************

In the next part of the lab, you will collect many samples to learn more about how sample proportions and confidence intervals constructed based on those samples vary from one sample to another.

-   Obtain a random sample.
-   Calculate the sample proportion, and use these to calculate and store the lower and upper bounds of the confidence intervals.
-   Repeat these steps 50 times.

Open the .Rmd file titled "Confidence_Intervals_App.Rmd" in the shared Labs folder, and click the **Run Document** button at the top (where the **Knit Document** button is usually located). Use the interactive app that opens to answer some of the following question.


### Exercise 4

Given a sample size of 60, as well as 1000 bootstrap samples for each interval, and 50 confidence intervals constructed (the default values for the above app), what proportion of your confidence intervals include the true population proportion? Is this proportion exactly equal to the confidence level? If not, explain why. 

**************************************************


**************************************************

## Additional Practice


### Exercise 5a

Using code from the **infer** package and data from the one sample you created in exercise 3a (`samp50`), find a confidence interval using the **standard error method** for the proportion of US Adults who think climate change is affecting their local community with a 95% confidence level and interpret it.  Compare the intervals produced by the methods in parts 3a and 5a.


**************************************************


**************************************************

### Exercise 5b

Choose a different confidence level than 95%. Would you expect a confidence interval at this level to me wider or narrower than the confidence interval you calculated at the 95% confidence level? Explain your reasoning.

**************************************************

**************************************************



### Exercise 6

Using the app, experiment with different sample sizes and comment on how the widths of intervals change as sample size changes (increases and decreases).

**************************************************

**************************************************




### Exercise 7
    
Finally, given a fixed sample size (say, 50), how does the width of the interval change as you increase the number of bootstrap samples? Support your answer with graphics and/or calculations. **Hint:** Does changing the number of bootstap samples affect the standard error?

**************************************************


**************************************************






### Exercise 8


A new study provides some evidence that playing action video games strengthens a person's ability to translate sensory information quickly into accurate decisions. Researchers had 23 volunteers with an average age of 20 look at moving arrays on a computer screen and indicate the direction in which the dots were moving. About half of the volunteers (11 individuals) reported playing action video games at least 5 times a week for the previous year, while the other 12 reported no video game playing in the previous year. The *response time* and *accuracy score* were both measured. A 95% confidence interval for the mean response time for game players minus the mean response time for non-players was -1.8 to -1.2 seconds, while a 95% confidence interval for mean accuracy score for game players minus mean accuracy score for non-players was -4.2 to +4.8 seconds.

A. Interpret the meaning of the 95% confidence interval for difference in mean response time.

*************************************************

************************************************* 

B. Is it likely that game players and non-game players are essentially the same in response time? Why or why not?

*************************************************


************************************************* 

C. Interpret the meaning of the 95% confidence interval for difference in mean accuracy score.

*************************************************


************************************************* 

D. Is it likely that game players and non-game players are essentially the same in accuracy? Why or why not?

*************************************************


************************************************* 


### Exercise 9

Suppose that a student is working on a statistics project using data on pulse rates collected from a random sample of 100 students from Reed college, and finds a 95% confidence interval for mean pulse rate to be 65.5 to 71.8. Discuss how each of the statements below would indicate an *improper* interpretation of this interval:

A. "I am 95% sure that all students will have pulse rates between 65.5 and 81.8 beats per minute."

*************************************************

************************************************* 

B. "I am 95% sure that the mean pulse rate for this sample of students will fall between 65.5 and 71.8 beats per minute."

*************************************************


************************************************* 

C. "I am 95% sure that the confidence interval for the average pulse rate of all students at this college is the interval from 65.5 to 71.8 beats per minute."

*************************************************


************************************************* 

D. "I am sure that 95% of all students at this college will have pulse rates between 65.5 and 71.8 beats per minute."

*************************************************


************************************************* 

E. "Of random samples of this size taken from students at Reed college, I am sure that 95% will have mean pulse rates between 65.5 and 71.8 beats per minute."

*************************************************


************************************************* 


### Exercise 10

In a recent study, 23 rats showed compassion that surprised scientists. Twenty-three of the 30 rats in the study freed another trapped rat in their cage, even when chocolate served as a distraction and even when the rats would then have to share the chocolate with their freed companion. (Rats, it turns out, love chocolate.) Rats did not open the cage when it was empty or when there was a stuffed animal inside, only when a fellow rat was trapped. We wish to use the sample to estimate the proportion of rats that show empathy in this way.

a.   What population parameter do researchers wish to investigate? What is the sample statistic will they use to estimate the population parameter, and what is its value?

*********************************************************************************


*********************************************************************************



b.	Describe how you could use 30 slips of paper to create **one** bootstrap statistic. Be specific.

*********************************************************************************



*********************************************************************************

c.	Of course, it would it would be extraodrinarily tedious to construct bootstrap distribution by hand, so let's use R. Run the following code chunk to produce the sample of 30 observations that the researchers collected.

```{r}
# Create a dataframe from the sample data
rats <- data.frame(empathy = c(rep("Yes", 23), rep("No", 7)))
```

Now, create a bootstrap distribution using 10,000 bootstrap statistics.

*********************************************************************************



*********************************************************************************


d. Plot the bootstrap distribution and use the plot to describe the center and shape of the distribution.

*********************************************************************************



*********************************************************************************

e. Explain why it is appropriate to use the percentile bootstrap method for confidence intervals in this case.

*********************************************************************************


*********************************************************************************

f. Construct a 95% confidence interval for the paramter of interest using the percentile method.

*********************************************************************************
 

*********************************************************************************
 

 
g.	Is there evidence that a majority of rats will show empathy?  Justify your answer.

*********************************************************************************



*********************************************************************************

 
### Exercise 11

The previous problem describes a study in which rats showed compassion by freeing a trapped rat. In the study, all six of six female rats showed compassion by freeing the trapped rat, while 17 of the 24 male rats did so. Researchers are interested in determining if it is plausible that male and female rats are equally compassionate based on this data.

To begin, run the following code chunk to produce the sample of 30 observations that the researchers collected.

```{r}
# Create a dataframe from the sample data
mfrats <- data.frame(empathy = c(rep("Yes", 23), rep("No", 7)), gender = c( rep("F", 6), rep("M", 17), rep("M", 7)     ))
```

a.   What population(s) are the researchers interested in? What population parameter(s) do the researchers wish to investigate? What is/are the sample statistic(s) they will use to estimate the population parameter(s)? What is/are the value(s) of these statistics?

*********************************************************************************


*********************************************************************************

b.	Describe how you could use 30 slips of paper to create **one** bootstrap statistic. Be sure to note any differences in your method here from the method you used in problem 1(b).

*********************************************************************************


*********************************************************************************

c.  Use R to a bootstrap distribution using 1,000 bootstrap statistics.

*********************************************************************************


*********************************************************************************


d. Plot the bootstrap distribution and use the plot to describe the center and shape of the distribution.

*********************************************************************************


*********************************************************************************

e. Based on your answer to part (d), do you have any concerns about using the standard error method bootstrap method for confidence intervals?

*********************************************************************************


*********************************************************************************

f. Construct a 99% confidence interval for the paramter of interest using the standard error method (use `infer` to construct your interval).

*********************************************************************************
 


*********************************************************************************
 

 
g.	Is there evidence that male and female rats are not equally compassionate?  Justify your answer.

*********************************************************************************



*********************************************************************************



### Exercise 12


Fuel is expensive. We want to compare the miles per gallon (MPG) of family sedans to the MPG of sports utility vehicles (SUV) to see if family sedans really do get better gas mileage (on average). I obtained the following sample of cars (in 2011) from the website http://www.fueleconomy.gov/feg/findacar.shtml#findacar.

```{r, message=FALSE}
gasmileage <- read_csv("/home/courses/math141s20/Data/gasmileage.csv")
```


a.	What are the parameters of interest? 


*********************************************************************************



*********************************************************************************

b.	 Create a useful graph of miles per gallon by type of car.  What can you infer from the graphic?


*********************************************************************************

*********************************************************************************


c. Using either bootstrap method, construct a 90% confidence interval for the difference in mean gas mileage.  

*********************************************************************************

*********************************************************************************


d. Suppose we wish to study the difference in **medians** instead of the difference in means. Using either bootstrap method, construct a 90% confidence interval for the difference in median gas mileage.  

*********************************************************************************


*********************************************************************************

e. As we discussed in class, the accuracy of these bootstrap methods requires that the bootstrap distribution of the statistic is fairly bell-shaped and symmetric.  Graph the bootstrap distribution of the difference in sample means and the bootstrap distribution of the difference in sample medians.  Are they both roughly bell-shaped and symmetric?  If one is not, why do you think that is?



*********************************************************************************


*********************************************************************************






