```{r setup, include=FALSE}
knitr::opts_chunk$set(echo = TRUE)
```

**Due Date:** Monday 09/13/2021, 11:59 PM

**Instructions:** <br>
 
 * Please provide complete answers/solutions for each question/problem. 
 
 * If it involves mathematical computations, please provide your reasoning and/or detailed solutions.
 
 * There are two ways you can write your answers, a: by handwriting (either physically or digitally), or b: by typing on a template document with file type options, Word or RMarkdown, which can be downloaded from the [course website](https://reed-statistics.github.io/math141-fall2021/homeworks.html){target="_blank"}.
 
 * If you had handwritten your answers/solutions on a physical paper, make sure to label it properly and please scan your document using a scanner app for convenience. Suggestions: (1) ["Tiny Scanner" for Android](https://play.google.com/store/apps/details?id=com.appxy.tinyscanner&hl=en_US&gl=US){target="_blank"} or (2) ["Scanner App" for iOS](https://apps.apple.com/us/app/scanner-app-scan-pdf-document/id595563753){target="_blank"}.
 
 * If you have questions or concerns, please feel free to ask the instructor.
 
 * **Please save your work as pdf file(s), don't put your name in any part of the document, and submit it to the Gradescope page for this course. Your document upload will correspond to your name automatically in Gradescope.**

\newpage

```{r, include = FALSE}
# NOTE: The file _common.R and the majority of the code snippets below are from the OpenIntro: IMS Github repository https://github.com/OpenIntroStat/ims/blob/master/_common.R
source("_common.R")
```

## I. Exploring Categorical Data

I.A. Consider the contingency table for homeownership and loan status shown in Table \@ref(tab:loan-home-loan-stat-totals).

```{r}
loans <- loans_full_schema %>%
  mutate(homeownership = as.character(homeownership)) %>%
  filter(homeownership != "") %>%
  mutate(
    loan_status    = tolower(loan_status), 
    loan_status    = fct_relevel(loan_status, "charged off", "current", "fully paid", "in grace period", "late (16-30 days)", "late (31-120 days)"), 
    homeownership = tolower(homeownership),
    homeownership = fct_relevel(homeownership, "rent", "mortgage", "own")
    )
```

1. Create a *row proportions* table using Table \@ref(tab:loan-home-loan-stat-totals).

2. Create a *column proportions* table using Table \@ref(tab:loan-home-loan-stat-totals).
  
3. What is the percentage of late (31-120 days) payments among renters?

4. What is the percentage of fully paid loans given that the homes are owned?

5. What is the percentage of mortgages given that they are in grace period?

```{r loan-home-loan-stat-totals}
loans %>%
  count(homeownership, loan_status) %>%
  pivot_wider(names_from = loan_status, values_from = n) %>% 
  adorn_totals(where = c("row", "col")) %>%
  kbl(linesep = "", booktabs = TRUE, caption = "A contingency table for homeownership and loan status") %>%
  kable_styling(bootstrap_options = c("striped", "condensed"), 
                latex_options = c("striped", "hold_position"), full_width = FALSE) %>%
  add_header_above(c(" " = 1, "Loan status" = 6, " " = 1)) %>%
  column_spec(2:8,width = "4em")
```

\newpage

I.B. Consider the barplots for homeownership and loan status shown in Fig. \@ref(fig:loan-home-loan-stat-bar-plot).

1. Identify each barplot according to type (stacked, dodged, standardized).

2. Which type of visualization is helpful in understanding the fraction of homeownership in each level of loan status?

3. Which type of visualization clearly displays that "current" loan status is the most common?

4. Which type of visualization shows that mortgage homeownership type is the most common among "current" loan status?

5. Which type of visualization shows that approximately half of late payments are renters?

```{r include=FALSE}
colors <- c(IMSCOL["blue", "full"], IMSCOL["yellow", "full"],
            IMSCOL["red", "full"], IMSCOL["green", "full"])
```

```{r loan-home-loan-stat-bar-plot, fig.cap = "Three bar plots displaying homeownership and loan status variables.", fig.asp = 0.8, fig.width = 18, fig.pos="H"}
p_stacked <- ggplot(loans, aes(x = loan_status, fill = homeownership)) +
  geom_bar() +
  scale_fill_manual(values = colors) +
  labs(x = "Loan status", y = "Count", fill = "Homeownership") + 
  theme(text = element_text(size = 30),
        axis.text.x = element_text(angle=90, hjust=1))

p_dodged <- ggplot(loans, aes(x = loan_status, fill = homeownership)) +
  geom_bar(position = "dodge") +
  scale_fill_manual(values = colors) +
  labs(x = "Loan status", y = "Count", fill = "Homeownership") + 
  theme(text = element_text(size = 30),
        axis.text.x = element_text(angle=90, hjust=1))

p_standardized <- ggplot(loans, aes(x = loan_status, fill = homeownership)) +
  geom_bar(position = "fill") +
  scale_fill_manual(values = colors) +
  labs(x = "Loan status", y = "Proportion", fill = "Homeownership") + 
  theme(text = element_text(size = 30),
        axis.text.x = element_text(angle=90, hjust=1))

p_stacked + p_dodged + p_standardized + guide_area() +
  plot_annotation(tag_levels = "A") +
  plot_layout(guides = "collect")
```

\newpage

## II. Exploring Numerical Data

II.A. Consider the scatterplot shown in Fig. \@ref(fig:median-hh-income-poverty).

1. What is the relationship between unemployment rate and poverty rate - is it linear or non-linear? Explain why.

2. Would you consider the trend to be positively or negatively associated? Explain why.

3. Are there any potential outliers in the scatterplot and which variable do the outliers belong to? Explain how these outliers might affect the statistical model fitted into the data (shown in dashed line).

```{r median-hh-income-poverty, fig.cap = "A scatterplot of the unemployment rate against the poverty rate for the county dataset. A statistical model has also been fit to the data and is shown as a dashed line.", fig.pos="H"}
ggplot(county, aes(x = unemployment_rate/100, y = poverty/100)) +
  geom_point(alpha = 0.3, fill = IMSCOL["black", "full"], 
             shape = 21, size = 3) +
  geom_smooth(method = "glm", linetype = "dashed", color = IMSCOL["red", "full"], se = FALSE) +
  labs(x = "Unemployment rate",y = "Poverty rate") +
  scale_x_continuous() +
  scale_y_continuous()
```

\newpage

II.B. Consider the number of observations per education level shown in Table \@ref(tab:observations-table-per-education), the summary statistics shown in Table \@ref(tab:summary-stats-per-education), and the histograms and boxplots shown in Fig. \@ref(fig:unemployment-split-by-education). 

1. Compute the interquartile Range (IQR) for each education level. What does this number represent and where does it show in the boxplots? Explain why we can't compute the IQR of the below_hs variable.

2. Using the mean and median, describe the skewness of the distributions, and which distribution is the most skewed. How many modes does each distribution have?

3. What information does the box plots provide that the histograms does not - and vice versa? Give a general answer and a specific answer related to the data showed.

```{r observations-table-per-education}
county %>%
  na.omit(median_edu) %>%
  count(median_edu) %>%
  pivot_wider(names_from = median_edu, values_from = n) %>% 
  adorn_totals(where = c("col")) %>%
  kbl(linesep = "", booktabs = TRUE, caption = "Number of observations per education level") %>%
  kable_styling(bootstrap_options = c("striped", "condensed"), 
                latex_options = c("striped", "hold_position"), full_width = FALSE) %>%
  add_header_above(c("Education level" = 5)) %>%
  column_spec(1:5, width = "5em")
```

```{r summary-stats-per-education}
# mode function only works on unimodal distributions
getmode <- function(v) {
   uniqv <- unique(v)
   uniqv[which.max(tabulate(match(v, uniqv)))]
}
county %>%
  na.omit(median_edu) %>%
  group_by(median_edu) %>% 
  summarize(min = min(unemployment_rate),
            q1 = quantile(unemployment_rate, 0.25),
            mode = getmode(unemployment_rate),
            median = median(unemployment_rate),
            mean = mean(unemployment_rate),
            q3 = quantile(unemployment_rate, 0.75),
            max = max(unemployment_rate)) %>%
  kbl(linesep = "", booktabs = TRUE, caption = "A summary statistics table by education level") %>%
  kable_styling(bootstrap_options = c("striped", "condensed"), 
                latex_options = c("striped", "hold_position"), full_width = FALSE) %>%
  add_header_above(c("Education level" = 1, "Summary statistics" = 7)) %>%
  column_spec(1:8, width = "3em")
```

```{r unemployment-split-by-education, fig.cap = "Histograms (Plot A) and side by-side box plots (Plot B) for unemployment rates split by education level", fig.width = 8, fig.asp = 0.7, fig.pos="H"}

p_hist <- county %>%
  na.omit(median_edu) %>%
  ggplot(aes(x = unemployment_rate, fill = median_edu)) + 
  geom_histogram(bins=50, alpha = 0.5) +
  scale_x_continuous() +
  labs(x = "Unemployment rate", y = NULL, fill = "Education level", color = "Education level") + 
  theme(text = element_text(size = 18))

p_box <- county %>%
  na.omit(median_edu)%>%
  ggplot(aes(x = unemployment_rate, y = median_edu, fill=median_edu)) +
  geom_boxplot(color="black") +
  scale_x_continuous() +
  labs(x = "Unemployment rate", y = NULL, fill = "Education level", color = "Education level") + 
  theme(text = element_text(size = 18))

p_hist / p_box + plot_annotation(tag_levels = "A")
```

\newpage

## III. Textbook Exercises

**Note:** To view the selected exercises below, please refer to the textbook, *[OpenIntro: Introduction to Modern Statistics (2021)](https://www.openintro.org/book/ims/){target="_blank"} by Mine Çetinkaya-Rundel and Johanna Hardin, First Edition.*

### [Section 4.8](https://openintro-ims.netlify.app/explore-categorical.html#chp4-exercises){target="_blank"}: Choose any 2 exercise problems below to answer.

**2.** Views on immigration.

**4.** Raise taxes.

**6.** Shipping holiday gifts data display.

<br>

### [Section 5.10](https://openintro-ims.netlify.app/explore-numerical.html#chp5-exercises){target="_blank"}: Choose any 6 exercise problems below to answer.

**8.** Medians and IQRs.

**10.** Histograms and box plots.

**12.** Median vs. mean.

**14.** Facebook friends.

**16.** Distributions and appropriate statistics.

**18.** Exam scores.

**20.** Oscar winners.

**22.** Income at the coffee shop.

**24.** Commute times.

**26.** NYC marathon winners.

<br>
