wrangling-presentation.Rpres

<style>
.small-code pre code {
  font-size: 1em;
}
</style>

wrangling
========================================================
author: Ana I. Bento & John M. Drake
date: 
autosize: true

Reading in data (files and objects)
========================================================

Files: You've already seen the main command for reading in data from files -> `read.csv`

We can also use `read_csv` in much the same way. It's slightly faster and the resulting data frame is recognized as a tibble (more info: https://cran.r-project.org/web/packages/tibble/vignettes/tibble.html)



Reading in data (files and objects)
========================================================


Objects: We can read previously saved objects (incl. data frames) with the `load` command

For example, if you created a data frame called `df` that took a lot of time (e.g. from some slow simulation) then you could first save it:

- `save(df,file='get_df.Rda')`

..and later, load it again (e.g. when you start a new session and it's no longer in memory)

- `load('get_df.Rda')`

The file extension .Rda is short for Rdata (and you can use .Rdata as the extension, if you prefer)


The 'tidy data' format
========================================================

It is good practice to work with 'tidy data':

- Each variable has its own column
- Each observation has its own row
- Each value has its own cell


This involves a few commands from the `dplyr` package, particularly:

- gather
- select
- mutate



The 'tidy data' format
========================================================



Additionally, we often make use of string manipulation (`stringr` package), especially:

- str_replace_all
- paste (base R)

Otherwise, if someone takes the temperature in saint augustine, and someone counts the number of people on the beach at St. Augustine - how will their data ever be combined?

FIPS codes
========================================================

State and County-level US data are often referenced by FIPS codes (a bit like zip codes)

- California = 6
- Orange County (CA) = 59
- Full FIPS = 6059

- Georgia = 13
- Clarke County (GA) = 59
- Full FIPS = 13059



Piping
========================================================
**A) Run several functions**

```{r, eval=F}
foo_foo <- hop(foo_foo, through = forest)
foo_foo <- scoop(foo_foo, up = field_mice)
foo_foo <- bop(foo_foo, on = head)
```

**B) String function calls together**

```{r, eval=F}
bop(
  scoop(
    hop(foo_foo, through = forest),
    up = field_mice
  ), 
  on = head
)
```

**C) Use piping**

```{r, eval=F}
foo_foo %>% hop(through = forest) %>% scoop(up = field_mouse) %>% bop(on = head)
```


Piping and assigning
========================================================

```{r}
x <- 2
y <- sqrt(sin(x))
y
```

```{r}
x <- 2
x <- sqrt(sin(x))
x
```



Piping and assigning
========================================================


```{r}
library(magrittr)
x <- 2
y <- x %>% sin %>% sqrt
y
```

```{r}
library(magrittr)
x <- 2
x %<>% sin %>% sqrt
x
```

Applying functions to each row
========================================================

```{r}
library(tidyverse)
library(magrittr)
pocket_money <- tibble(name=c("Jack","Jill"),
                       savings=c(10,15),chores=c(5,2)) %>% print
```

Applying functions to each row
========================================================

```{r}
get_sum<-function(savings,chores){return(sum(savings,chores))}
pocket_money %>% rowwise %>% mutate(total=get_sum(savings,chores)) %>% print
```

Applying functions to each row
========================================================


```{r}
get_sum<-function(savings,chores){return(sum(savings,chores))}
pocket_money %>% mutate(total=get_sum(savings,chores)) %>% print
```


Combining data sets
========================================================

In base R, data frames are combined using the `merge` function

Using the `dplyr` package, tibbles are combined using `join` functions (https://cran.r-project.org/web/packages/dplyr/vignettes/two-table.html)

The different types of `join` are best explored by example, like with these superheros -> http://stat545.com/bit001_dplyr-cheatsheet.html

You can specifiy the column names to match on, otherwise R will do a _natural join_ using all variables in common


Combining data sets
========================================================

```{r}
gourmet <- tibble(
  state=c("GA","CA","NC","TX"),
  sauce=c("sweetBBQ","hot","tangyBBQ","smokyBBQ")
)

greeting <- tibble(
  state=c("TX","GA","CA","NY"),
  word=c("howdy","hey y'all","'sup","hello")
)

full_join(gourmet,greeting)
```


Combining data sets
========================================================

```{r}
gourmet <- tibble(
  state=c("GA","CA","NC","TX"),
  sauce=c("sweetBBQ","hot","tangyBBQ","smokyBBQ")
)

greeting <- tibble(
  state=c("TX","GA","CA","NY"),
  word=c("howdy","hey y'all","'sup","hello")
)

inner_join(gourmet,greeting)
```

Obtaining summary information
========================================================

```{r}
set.seed(123)
grades <- tibble(
  student_ID=as.character(1:10),
  major=sample(c("E","B"),10,replace=T),
  U_G=sample(c("U","G"),10,replace=T),
  score=sample(60:100,10,replace=T)
) %>% print
```

Obtaining summary information
========================================================


```{r}
grades %>% group_by(major) %>% summarize(avScore=mean(score))

```