---
title: "Exercise 3.1 Solution"
author: "Put Your Name Here"
output:
  html_document:
    toc: true
---

<!-- You can knit this document at any time to see the draft of your answer. 
     An html file will be produced in the same folder as the Rmd file. -->

<!-- When you have completed the exercise, knit the document for the final time. -->

```{r setup, include = FALSE}
knitr::opts_chunk$set(
  echo = TRUE,
  comment = NA,
  fig.width = 7,
  fig.height = 4,
  dpi = 130
)
```

In this exercise, we will require the `tidyverse`, `knitr` and `janitor` packages. 

```{r packages, message = FALSE, warning = FALSE}
library(tidyverse)
library(knitr)
library(janitor)
library(gt)
```

The goal of this exercise is to try some of the data manipulation options available in R. The data we are looking at today are songs downloaded from Spotify via the `spotifyr` package. You can even use this package to download your own playlists!

### (a) Read in and explore the data set.

>The file is called `spotify_songs.csv` and you can read the file in using the `read_csv` function. Choose `songs` as the name of the data frame if you want to be consistent with the rest of the exercise and the solutions we provide. Explore the variables in RStudio or using code.  A detailed summary of each of the variables can be found here: https://github.com/rfordatascience/tidytuesday/blob/master/data/2020/2020-01-21/readme.md.

```{r import-data}
songs <- read_csv("spotify_songs.csv")
glimpse(songs)
```

### (b) Remove the playlist ID variable (`playlist_id`) from the data set.

> Use the `select()` function to remove the variable. This requires you to use a negative sign.
>
> Give this new data frame a new name.

```{r }
songs2 <- songs %>%
  select(-playlist_id)
```


### (c) Rename the variable for mode to be more explanatory

> Use the `rename()` function to rename the variable `mode` so that it indicates that it is a mode of the key in particular. 
>
> Create a new code chunk that includes the code from (b) as well, so that it can all be run together as one chunk.

```{r }
songs2 <- songs %>%
  select(-playlist_id) %>%
  rename(key_mode = mode)
```


### (d) Change the scale of the danceability variable to be out of 100

> The danceability variable is currently a number from 0 to 1.  Use the `mutate()` function to create a new percentage danceability from 0 to 100. 
>
> Include all the code in one chunk again, as you continue building on it.

```{r }
songs2 <- songs %>%
  select(-playlist_id) %>%
  rename(key_mode = mode) %>%
  mutate(dance100 = danceability*100)
```


### (e) Order the data from highest danceability to lowest danceability

> Use the `arrange()` function to order the data frame, on your new danceability scale. Recall that `desc()` within `arrange` uses descending order.
>
> What are some of the top songs for danceability?
>
> Add the existing code to this chunk as well like above.

```{r }
songs2 <- songs %>%
  select(-playlist_id) %>%
  rename(key_mode = mode) %>%
  mutate(dance100 = danceability*100) %>%
  arrange(desc(dance100))
songs2
```


### (f) Select only those songs with high danceability

> Use the `filter()` function to select songs with danceability greater than or equal to 90%.
>
> How many songs are in this category?  Use `tabyl` to see how many highly danceable songs are in each genre. **Be careful not to override the original data set with this table.** There is no need to assign this `tabyl` to an object unless you intend to refer to it later.
>
> It would be useful to know how the proportions of each genre compare to the entire set of songs as well. So instead of filtering by high danceability, try creating a binary variable that indicates high danceability (90 or more) compared with the alternative, using the `mutate()` function. You may like to save this version of the data set with a new name.


```{r }
songs %>%
  select(-playlist_id) %>%
  rename(key_mode = mode) %>%
  mutate(dance100 = danceability*100) %>%
  arrange(desc(dance100)) %>%
  filter(dance100>=90) %>%
  tabyl(playlist_genre) %>%
  adorn_pct_formatting() %>%
  gt()
```

There seem to be a large number of rap songs with high danceability.

```{r }
songs2 <- songs %>%
  select(-playlist_id) %>%
  rename(key_mode = mode) %>%
  mutate(dance100 = danceability*100) %>%
  arrange(desc(dance100)) %>%
  mutate(highdance = dance100>=90) %>%
  mutate(highdance = replace(highdance, highdance == FALSE, "low")) %>%
  mutate(highdance = replace(highdance, highdance == TRUE, "high"))

songs2 %>%
  tabyl(playlist_genre, highdance) %>%
  gt()
```

Use the `replace()` function within `mutate` to give this new variable a more informative level names (rather than the default `TRUE` or `FALSE`). From here we can see that not only are there a large number of rap songs with high danceability; the highest percentage of high danceability songs is also found in the rap genre.

Note also that we separate the data manipulation step from the `tabyl` step, so that the updated data (and not the table) are saved with the name `songs2`, which we can refer to below.


### (g) Produce some plots to summarise the relationships between some variables.

> Consider the relationship between popularity (`track_popularity`) and danceability for the full data set.  What visual display could you use to explore this relationship?
>
> Hint: there are a large number of data points, so there will be a lot of overlap in the points.  There are a few techniques that can help. For example, you can add additional information with a `geom_smooth()` curve. You can also summarise the points in each location with something like `geom_hex()` (requiring package `hexbin`), or make the points transparent (`geom_point(alpha = 0.1)`) or very small (`geom_point(shape= ".")`) Play around with these options to see what you think works best. Don't be surprised if these figures take some time to generate as there are a large number of data points for each.
>
> How might you explore this relationship for different genres (`playlist_genre`)?

```{r }
ggplot(songs2,
       aes(x = dance100,
           y = track_popularity)) +
  geom_point()
```

```{r }
ggplot(songs2,
       aes(x = dance100,
           y = track_popularity)) +
  geom_point() + geom_smooth()
```

```{r }
ggplot(songs2,
       aes(x = dance100,
           y = track_popularity)) +
  geom_point(alpha = 0.1) 
```

```{r }
ggplot(songs2,
       aes(x = dance100,
           y = track_popularity)) +
  geom_point(shape = ".") 
```

```{r }
library(hexbin)
ggplot(songs2,
       aes(x = dance100,
           y = track_popularity)) +
  geom_hex() 
```
Here are a range of options. For the genre figure, let's add colour and smoothed lines by genre.

```{r }
ggplot(songs2,
       aes(x = dance100,
           y = track_popularity,
           colour = playlist_genre)) +
  geom_point() + geom_smooth()
```

It is still a bit hard to see what is happening with all the genres on the same plot, so we will also try panelling by genre and add some transparency as well.

```{r }
ggplot(songs2,
       aes(x = dance100,
           y = track_popularity)) +
  geom_point(alpha = 0.1) + geom_smooth() +
  facet_wrap(vars(playlist_genre), nrow=3)
```

In general, there is not evidence of a strong relationship between danceability and popularity of a track, though there are some slight positive trends (for example, for rap).

### (h) Extension exercises.

> Consider some other relationships between variables that you are interested in and create the code to explore these, with summary tables and/or visual displays.
>
> Download the airport screening file used in lectures and perform some of your own data transformations and summaries of these variables.

---

&copy; 2022 Statistical Consulting Centre, The University of Melbourne.
