performr vignette
Silas Tittes
July 01, 2020
NOTE: This document is in progress. Feel free to suggest changes on the package’s GitHub page, or by email silas.tittes@gmail.com.
How to use performr package
performr implements a probabilistic Bayesian hierarchical model (using Stan) to predict tolerance/performance curves for a set of input taxa. The manuscript that describes the method is in revision. This page is intended to help those interested in using the model for their own data.
Installation
Installing performr is done using the following command:
#install
devtools::install_github("silastittes/performr", local = FALSE)
#load other libraries used below
library(performr)
library(tidyverse)
library(ggridges)
theme_set(theme_minimal())As the model is written in Stan, it requires a c++ compiler. For Apple and PC users, you likely need to install one. If the installation command above fails, Mac users can open their terminal and type:
xcode-select --installThis should open some dialogue to install the Xcode developer tools. Follow the prompts to install. Once the install finishes, try the install_github() command again. At this point, I have not tested the package on PCs, but I would guess a similar step is required (please email me if you have success on a PC!).
Linux users can likely run the command as-is, or by using devtools::install_github("silastittes/performr"), which would use the pre-compiled version that my Linux machine built (Linux Mint 18.3).
Last installation detail, performr depends on several packages. These can take some time to install, but usually goes smoothly as long as your R is up to date! If you run into troubles, check the R session info at the bottom of this page. Compare to your own session info for missing packages and older package versions.
Simulating example data
For demonstration and model validation, data can be simulated from the hyperpriors. The true parameter values for both hyper parameters and species-level parameters are returned in addition to a simulated data frame. All arguments have defaults, but fiddling with parameter values is encouraged.
I’ll demonstrate a data set with three species, 20 positions sampled along an environmental axis, and three replicates per species at each position along the axis (wouldn’t that be something):
set.seed(223524)
perf_df <- simulate_data(n_spp = 3, n_axis = 20, n_reps = 3)Here are a few quick ways to look at the output:
head(perf_df$sim_data)## # A tibble: 6 x 4
## x trait mu species
## <dbl> <dbl> <dbl> <int>
## 1 -4.72 0.0668 0.0389 1
## 2 -4.72 0.0471 0.0389 1
## 3 -4.72 0 0.0389 1
## 4 -4.32 0.0560 0.0983 1
## 5 -4.32 0.102 0.0983 1
## 6 -4.32 0.0726 0.0983 1str(perf_df)## List of 2
## $ true_params:List of 12
## ..$ shape1 : num [1:3] 3.84 3.17 2.25
## ..$ shape2 : num [1:3] 2.61 2.97 2.64
## ..$ stretch : num [1:3] 2.55 2.1 1.97
## ..$ x_min : num [1:3] -5.76 -5.39 -4.55
## ..$ x_max : num [1:3] 4.41 2.52 4.38
## ..$ nu : num [1:3] 17.1 33.6 26
## ..$ mu_shape1 : num 2.57
## ..$ mu_shape2 : num 2.91
## ..$ mu_stretch: num 1.75
## ..$ mu_min : num -5.39
## ..$ mu_max : num 4.22
## ..$ mu_nu : num 3.96
## $ sim_data : tibble [180 × 4] (S3: tbl_df/tbl/data.frame)
## ..$ x : num [1:180] -4.72 -4.72 -4.72 -4.32 -4.32 ...
## ..$ trait : num [1:180] 0.0668 0.0471 0 0.056 0.1024 ...
## ..$ mu : num [1:180] 0.0389 0.0389 0.0389 0.0983 0.0983 ...
## ..$ species: int [1:180] 1 1 1 1 1 1 1 1 1 1 ...perf_df$sim_data %>%
mutate(species = factor(species)) %>%
ggplot(aes(x, trait, colour = species)) +
geom_point() +
geom_line(aes(x, mu))
Simulated data for three ‘species’ plotted with the curves used to generate the samples.
There are 3 columns of input data required to run the model, where each row represents an individual unit of sampling (name and order the columns as you please):
Integers representing the groups (or “species”) each sample belongs to – even if there is only one group.
An x-axis consisting of measurements from some environmental variable (soil moisture, temperature, etc.).
A measure of performance for each individual studied in that environmental condition (trait). Rows with missing values must be removed before fitting –
tidyr::drop_na()is good for this.
Fitting the model with performr::stan_performance()
Now that we have some example data, we can fit the model. Arguments that include “pr” are parameters for the priors (for brevity, I will gloss over prior details here and refer readers to the paper once available), which I’ve set to match the defaults of the simulation (so this is best case scenario for the model getting it right). Notice, I’ve set a seed to make the results reproducible, and I’ve reduced the total number of iterations to 200. The default iterations of 2000 (or more) should be used for analysis, but 200 is good for demonstration and troubleshooting purposes.
perf_out <-
stan_performance(
df = perf_df$sim_data,
response = trait,
treatment = x,
group_ids = species,
shape1_pr_mu = 2,
shape1_pr_sig = 1,
shape2_pr_mu = 2,
shape2_pr_sig = 1,
stretch_pr_mu = 1,
stretch_pr_sig = 1,
nu_pr_shape = 8,
nu_pr_scale = 3,
min_pr_mu = -5,
min_pr_sig = 1,
max_pr_mu = 5,
max_pr_sig = 1,
iter = 200,
seed = 7345,
file_id = "stan_example"
)Although it won’t appear on the webpage (the code above is not being evaluated) Stan warns us about the max tree depth (it may also warn about divergent transitions if the simulation is modified). Ideally, those messages should not appear. You can modify the arguments it recommends in your call to stan_performance() as you would if calling Stan directly. Typically, a max_tree_depth between 12 to 14 works well (though slower). The adapt_delta defaults to 0.95, which is sufficient in my experience. Notice the file_id argument, which takes a file prefix for saving the model output to file in your current directory.
We can read in the model from file as such:
perf_stan <-
rstan::read_stan_csv(
csvfiles = c(
"stan_example.samples_1.csv",
"stan_example.samples_2.csv",
"stan_example.samples_3.csv",
"stan_example.samples_4.csv"
)
)Output
The output is the same as any Stan model.
knitr::kable(rstan::summary(perf_stan)$summary, digits = 3)| mean | se_mean | sd | 2.5% | 25% | 50% | 75% | 97.5% | n_eff | Rhat | |
|---|---|---|---|---|---|---|---|---|---|---|
| shape1[1] | 3.636 | 0.020 | 0.329 | 3.068 | 3.415 | 3.581 | 3.858 | 4.348 | 282.480 | 1.007 |
| shape1[2] | 3.345 | 0.009 | 0.184 | 3.005 | 3.229 | 3.336 | 3.451 | 3.732 | 453.234 | 1.008 |
| shape1[3] | 2.432 | 0.006 | 0.096 | 2.267 | 2.368 | 2.420 | 2.492 | 2.643 | 300.961 | 1.015 |
| shape2[1] | 3.318 | 0.058 | 0.733 | 2.212 | 2.785 | 3.194 | 3.723 | 5.119 | 161.212 | 1.013 |
| shape2[2] | 3.122 | 0.012 | 0.273 | 2.570 | 2.963 | 3.137 | 3.300 | 3.597 | 492.069 | 1.002 |
| shape2[3] | 3.329 | 0.046 | 0.600 | 2.404 | 2.891 | 3.290 | 3.744 | 4.508 | 166.263 | 1.041 |
| stretch[1] | 2.514 | 0.011 | 0.194 | 2.179 | 2.392 | 2.503 | 2.635 | 2.964 | 303.164 | 1.007 |
| stretch[2] | 2.135 | 0.003 | 0.064 | 2.017 | 2.087 | 2.136 | 2.174 | 2.265 | 463.008 | 1.001 |
| stretch[3] | 1.825 | 0.008 | 0.104 | 1.629 | 1.751 | 1.820 | 1.900 | 2.016 | 178.662 | 1.031 |
| min_max[1,1] | -5.398 | 0.021 | 0.362 | -6.159 | -5.619 | -5.341 | -5.152 | -4.812 | 295.698 | 1.001 |
| min_max[1,2] | 5.021 | 0.042 | 0.621 | 3.912 | 4.583 | 4.952 | 5.385 | 6.299 | 218.130 | 1.001 |
| min_max[2,1] | -5.533 | 0.009 | 0.174 | -5.902 | -5.634 | -5.524 | -5.412 | -5.236 | 409.407 | 1.008 |
| min_max[2,2] | 2.558 | 0.004 | 0.087 | 2.373 | 2.509 | 2.571 | 2.618 | 2.703 | 445.618 | 1.000 |
| min_max[3,1] | -4.676 | 0.004 | 0.081 | -4.864 | -4.730 | -4.666 | -4.620 | -4.537 | 444.753 | 1.009 |
| min_max[3,2] | 4.831 | 0.046 | 0.568 | 3.848 | 4.398 | 4.796 | 5.221 | 5.997 | 153.640 | 1.043 |
| nu[1] | 18.210 | 0.097 | 2.136 | 14.405 | 16.755 | 18.153 | 19.609 | 22.454 | 486.572 | 1.004 |
| nu[2] | 38.470 | 0.184 | 3.837 | 30.529 | 35.753 | 38.559 | 41.090 | 45.929 | 436.497 | 1.004 |
| nu[3] | 32.539 | 0.128 | 2.942 | 26.829 | 30.617 | 32.609 | 34.288 | 38.747 | 530.208 | 0.997 |
| mu_shape1 | 2.882 | 0.028 | 0.514 | 1.906 | 2.537 | 2.888 | 3.212 | 3.900 | 333.334 | 1.013 |
| mu_shape2 | 2.890 | 0.050 | 0.578 | 1.794 | 2.525 | 2.902 | 3.309 | 3.950 | 133.850 | 1.021 |
| mu_stretch | 1.852 | 0.032 | 0.511 | 0.883 | 1.510 | 1.850 | 2.227 | 2.788 | 251.485 | 1.001 |
| mu_min | -5.129 | 0.033 | 0.487 | -6.110 | -5.441 | -5.128 | -4.824 | -4.087 | 217.046 | 0.998 |
| mu_max | 4.326 | 0.048 | 0.539 | 3.244 | 3.979 | 4.334 | 4.699 | 5.339 | 124.908 | 1.031 |
| mu_nu | 0.295 | 0.002 | 0.053 | 0.206 | 0.257 | 0.293 | 0.328 | 0.406 | 520.257 | 0.993 |
| x_min[1] | -5.398 | 0.021 | 0.362 | -6.159 | -5.619 | -5.341 | -5.152 | -4.812 | 295.698 | 1.001 |
| x_min[2] | -5.533 | 0.009 | 0.174 | -5.902 | -5.634 | -5.524 | -5.412 | -5.236 | 409.407 | 1.008 |
| x_min[3] | -4.676 | 0.004 | 0.081 | -4.864 | -4.730 | -4.666 | -4.620 | -4.537 | 444.753 | 1.009 |
| x_max[1] | 5.021 | 0.042 | 0.621 | 3.912 | 4.583 | 4.952 | 5.385 | 6.299 | 218.130 | 1.001 |
| x_max[2] | 2.558 | 0.004 | 0.087 | 2.373 | 2.509 | 2.571 | 2.618 | 2.703 | 445.618 | 1.000 |
| x_max[3] | 4.831 | 0.046 | 0.568 | 3.848 | 4.398 | 4.796 | 5.221 | 5.997 | 153.640 | 1.043 |
| lp__ | -13.662 | 0.321 | 3.795 | -21.247 | -16.072 | -13.385 | -10.956 | -7.699 | 140.101 | 1.039 |
We can access the posterior draws using rstan::extract(), which produces a list containing draws for each parameter of the model.
draws <- rstan::extract(perf_stan)
ndraws <- length(draws$lp__)There are a lot of good packages out there for visualizing posteriors. I haven’t learned any of them yet, but here’s a quick approach I’m fond of:
#choose parameter
param <- "x_max"
max_df <- draws[[param]] %>%
as_tibble() %>%
gather(species, value)## Warning: The `x` argument of `as_tibble.matrix()` must have column names if `.name_repair` is omitted as of tibble 2.0.0.
## Using compatibility `.name_repair`.
## This warning is displayed once every 8 hours.
## Call `lifecycle::last_warnings()` to see where this warning was generated.max_df %>%
ggplot(aes(value, species, fill = species)) +
geom_density_ridges(colour = "white") +
xlab(param) +
theme(legend.position = "none")## Picking joint bandwidth of 0.113
Quickly, let’s eyeball how these marginal posteriors for the x_max parameter (where performance falls to zero above the optimum) for each species.
max_df %>%
group_by(species) %>%
summarise(
cred_025 = quantile(value, 0.025),
cred_975 = quantile(value, 0.975)
) %>%
mutate(
truth = perf_df$true_params$x_max
) %>%
mutate(
within_cred = truth > cred_025 & truth < cred_975
)## `summarise()` ungrouping output (override with `.groups` argument)## # A tibble: 3 x 5
## species cred_025 cred_975 truth within_cred
## <chr> <dbl> <dbl> <dbl> <lgl>
## 1 V1 3.91 6.30 4.41 TRUE
## 2 V2 2.37 2.70 2.52 TRUE
## 3 V3 3.85 6.00 4.38 TRUECool, looks like the 95% credible interval captures the true value in each case. It would be easy to extend this approach using a function and quickly verify each parameter. Notice the increased uncertainty in posteriors further from the treatments on the x-axis … Love that!
Visualize uncertainty in curves predictions
First, let’s generate a tidy data frame with all the parameters, plus some valuable derived parameters, like the optimum, area, and breadth for each species. We will use this data frame for other tasks below as well.
head(
tidy_perf <-
perform_df(
perf_stan,
species_order = c(1, 2, 3)
)
)## # A tibble: 6 x 12
## # Groups: species [1]
## species x_min draw x_max shape1 shape2 stretch nu maxima breadth area
## <chr> <dbl> <int> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>
## 1 1 -5.39 1 6.28 3.56 5.16 1.88 14.6 1.42 11.7 21.9
## 2 1 -4.87 2 4.37 3.20 2.39 2.51 18.3 1.66 9.24 23.2
## 3 1 -5.18 3 5.30 3.35 3.32 2.26 16.1 1.60 10.5 23.7
## 4 1 -5.36 4 6.20 3.72 4.76 2.61 23.0 1.74 11.6 30.1
## 5 1 -5.08 5 5.19 3.21 2.87 2.38 17.1 1.74 10.3 24.5
## 6 1 -5.66 6 5.43 3.87 4.13 2.33 17.4 1.58 11.1 25.9
## # … with 1 more variable: special <dbl>The next step is to generate prediction intervals using the predict_interval() function. The function generates posterior quantiles for each set of posterior draws specified (x_draws), and averages over the quantiles. The argument p can takes a vector of credible levels, which can be modified to consider other and/or additional levels.
#set up sequence from smallest to largest x
x_seq = seq(min(draws$x_min),
max(draws$x_max),
length.out = 100)
#sub-sample draws randomly
poly_draws <- sample(1:ndraws, 100)
head(
creds <-
predict_interval(
x = x_seq,
spp = spp,
par_df = tidy_perf,
x_draws = poly_draws,
p = c(0.95, 0.5))
)## # A tibble: 6 x 6
## # Groups: species [1]
## species x lower upper mu level
## <chr> <dbl> <dbl> <dbl> <dbl> <dbl>
## 1 1 -6.67 0 0.110 0 95
## 2 1 -6.67 0 0.0379 0 50
## 3 1 -6.52 0 0.110 0 95
## 4 1 -6.52 0 0.0379 0 50
## 5 1 -6.38 0 0.110 0 95
## 6 1 -6.38 0 0.0379 0 50Notice the output of predict_interval() also produces a “mu” column, which contains the mean preduction of the curve for each input species.
Here is an example of how the data could be plotted. Other approach using plot() are certainly possible:
ggplot(data = filter(creds, level == 95)) +
geom_ribbon(
aes(x = x,
ymin = upper,
ymax = lower,
fill = species),
alpha = 0.3,
inherit.aes = F) +
geom_ribbon(
data = filter(creds, level == 50),
aes(x = x,
ymin = upper,
ymax = lower,
fill = species),
inherit.aes = F,
alpha = 0.7) +
geom_line(aes(x, mu, group = species), inherit.aes = F, colour = "white", lwd = 1) +
geom_point(
data = perf_df$sim_data,
aes(x,
trait,
alpha = 0.5),
inherit.aes = F,
colour = "grey70") +
theme(legend.position = "none")
Posterior predictive data generation
The whole concept of posterior prediction and generating data from the model is one of my favorite things about Bayesian statistics. It allows for a lot of creativity in how we check our assumptions. We can generate data from the model using this approach:
Now we can generate pseudo observed data from the posterior draws.
tidy_perf %>%
filter(draw == 1) %>%
posterior_predict(par_df = .) %>%
head()## # A tibble: 6 x 5
## x trait mu species draw
## <dbl> <dbl> <dbl> <chr> <int>
## 1 -5.58 0 0 1 1
## 2 -5.46 0 0 1 1
## 3 -5.34 0 0.0000387 1 1
## 4 -5.22 0.0161 0.000741 1 1
## 5 -5.10 0 0.00282 1 1
## 6 -4.98 0 0.00674 1 1By default, the x-axis will be constructed as a sequence of 100 values that start from the minimum posterior draw value of x_min and end at the maximum of x_max. This default can be changed using the x argument. Filtering for posterior draws is not required before using the function – posterior predictive data will be produced for every posterior draw passed to par_df.
If you have questions, comments, or concerns, please get in touch. I would be excited to discuss!
<button data-toggle=“collapse” data-target=“#sessioninfo” class="btn btn-primary btn-md btn-info> R session info
#writeLines(readLines(file.path(Sys.getenv("HOME"), ".R/Makevars")))
devtools::session_info()## ─ Session info ───────────────────────────────────────────────────────────────
## setting value
## version R version 4.0.0 (2020-04-24)
## os macOS Catalina 10.15.5
## system x86_64, darwin17.0
## ui X11
## language (EN)
## collate en_US.UTF-8
## ctype en_US.UTF-8
## tz America/Denver
## date 2020-07-01
##
## ─ Packages ───────────────────────────────────────────────────────────────────
## package * version date lib source
## assertthat 0.2.1 2019-03-21 [1] CRAN (R 4.0.0)
## backports 1.1.8 2020-06-17 [1] CRAN (R 4.0.0)
## bindr * 0.1.1 2018-03-13 [1] CRAN (R 4.0.0)
## blob 1.2.1 2020-01-20 [1] CRAN (R 4.0.0)
## broom 0.5.6 2020-04-20 [1] CRAN (R 4.0.0)
## callr 3.4.3 2020-03-28 [1] CRAN (R 4.0.0)
## cellranger 1.1.0 2016-07-27 [1] CRAN (R 4.0.0)
## cli 2.0.2 2020-02-28 [1] CRAN (R 4.0.0)
## codetools 0.2-16 2018-12-24 [1] CRAN (R 4.0.0)
## colorspace 1.4-1 2019-03-18 [1] CRAN (R 4.0.0)
## crayon 1.3.4 2017-09-16 [1] CRAN (R 4.0.0)
## DBI 1.1.0 2019-12-15 [1] CRAN (R 4.0.0)
## dbplyr 1.4.4 2020-05-27 [1] CRAN (R 4.0.0)
## desc 1.2.0 2018-05-01 [1] CRAN (R 4.0.0)
## devtools 2.3.0 2020-04-10 [1] CRAN (R 4.0.0)
## digest 0.6.25 2020-02-23 [1] CRAN (R 4.0.0)
## dplyr * 1.0.0 2020-05-29 [1] CRAN (R 4.0.0)
## ellipsis 0.3.1 2020-05-15 [1] CRAN (R 4.0.0)
## evaluate 0.14 2019-05-28 [1] CRAN (R 4.0.0)
## fansi 0.4.1 2020-01-08 [1] CRAN (R 4.0.0)
## farver 2.0.3 2020-01-16 [1] CRAN (R 4.0.0)
## forcats * 0.5.0 2020-03-01 [1] CRAN (R 4.0.0)
## fs 1.4.1 2020-04-04 [1] CRAN (R 4.0.0)
## generics 0.0.2 2018-11-29 [1] CRAN (R 4.0.0)
## ggplot2 * 3.3.2 2020-06-19 [1] CRAN (R 4.0.0)
## ggridges * 0.5.2 2020-01-12 [1] CRAN (R 4.0.0)
## glue 1.4.1 2020-05-13 [1] CRAN (R 4.0.0)
## gridExtra 2.3 2017-09-09 [1] CRAN (R 4.0.0)
## gtable 0.3.0 2019-03-25 [1] CRAN (R 4.0.0)
## haven 2.3.1 2020-06-01 [1] CRAN (R 4.0.0)
## highr 0.8 2019-03-20 [1] CRAN (R 4.0.0)
## hms 0.5.3 2020-01-08 [1] CRAN (R 4.0.0)
## htmltools 0.4.0 2019-10-04 [1] CRAN (R 4.0.0)
## httr 1.4.1 2019-08-05 [1] CRAN (R 4.0.0)
## inline 0.3.15 2018-05-18 [1] CRAN (R 4.0.0)
## jsonlite 1.6.1 2020-02-02 [1] CRAN (R 4.0.0)
## knitr 1.28 2020-02-06 [1] CRAN (R 4.0.0)
## labeling 0.3 2014-08-23 [1] CRAN (R 4.0.0)
## lattice 0.20-41 2020-04-02 [1] CRAN (R 4.0.0)
## lifecycle 0.2.0 2020-03-06 [1] CRAN (R 4.0.0)
## loo 2.2.0 2019-12-19 [1] CRAN (R 4.0.0)
## lubridate 1.7.9 2020-06-08 [1] CRAN (R 4.0.0)
## magrittr * 1.5 2014-11-22 [1] CRAN (R 4.0.0)
## matrixStats 0.56.0 2020-03-13 [1] CRAN (R 4.0.0)
## memoise 1.1.0 2017-04-21 [1] CRAN (R 4.0.0)
## modelr 0.1.8 2020-05-19 [1] CRAN (R 4.0.0)
## munsell 0.5.0 2018-06-12 [1] CRAN (R 4.0.0)
## nlme 3.1-148 2020-05-24 [1] CRAN (R 4.0.0)
## performr * 0.2 2020-07-01 [1] Github (silastittes/performr@fb5b84d)
## pillar 1.4.4 2020-05-05 [1] CRAN (R 4.0.0)
## pkgbuild 1.0.8 2020-05-07 [1] CRAN (R 4.0.0)
## pkgconfig 2.0.3 2019-09-22 [1] CRAN (R 4.0.0)
## pkgload 1.1.0 2020-05-29 [1] CRAN (R 4.0.0)
## plyr 1.8.6 2020-03-03 [1] CRAN (R 4.0.0)
## prettyunits 1.1.1 2020-01-24 [1] CRAN (R 4.0.0)
## processx 3.4.2 2020-02-09 [1] CRAN (R 4.0.0)
## ps 1.3.3 2020-05-08 [1] CRAN (R 4.0.0)
## purrr * 0.3.4 2020-04-17 [1] CRAN (R 4.0.0)
## R6 2.4.1 2019-11-12 [1] CRAN (R 4.0.0)
## Rcpp * 1.0.4.6 2020-04-09 [1] CRAN (R 4.0.0)
## RcppParallel 5.0.2 2020-06-24 [1] CRAN (R 4.0.0)
## readr * 1.3.1 2018-12-21 [1] CRAN (R 4.0.0)
## readxl 1.3.1 2019-03-13 [1] CRAN (R 4.0.0)
## remotes 2.1.1 2020-02-15 [1] CRAN (R 4.0.0)
## reprex 0.3.0 2019-05-16 [1] CRAN (R 4.0.0)
## rlang 0.4.6 2020-05-02 [1] CRAN (R 4.0.0)
## rmarkdown 2.2 2020-05-31 [1] CRAN (R 4.0.0)
## rprojroot 1.3-2 2018-01-03 [1] CRAN (R 4.0.0)
## rstan 2.19.3 2020-02-11 [1] CRAN (R 4.0.0)
## rstantools 2.0.0 2019-09-15 [1] CRAN (R 4.0.0)
## rstudioapi 0.11 2020-02-07 [1] CRAN (R 4.0.0)
## rvest 0.3.5 2019-11-08 [1] CRAN (R 4.0.0)
## scales 1.1.1 2020-05-11 [1] CRAN (R 4.0.0)
## sessioninfo 1.1.1 2018-11-05 [1] CRAN (R 4.0.0)
## StanHeaders 2.21.0-5 2020-06-09 [1] CRAN (R 4.0.0)
## stringi 1.4.6 2020-02-17 [1] CRAN (R 4.0.0)
## stringr * 1.4.0 2019-02-10 [1] CRAN (R 4.0.0)
## testthat 2.3.2 2020-03-02 [1] CRAN (R 4.0.0)
## tibble * 3.0.1 2020-04-20 [1] CRAN (R 4.0.0)
## tidyr * 1.1.0 2020-05-20 [1] CRAN (R 4.0.0)
## tidyselect 1.1.0 2020-05-11 [1] CRAN (R 4.0.0)
## tidyverse * 1.3.0 2019-11-21 [1] CRAN (R 4.0.0)
## truncnorm * 1.0-8 2018-02-27 [1] CRAN (R 4.0.0)
## usethis 1.6.1 2020-04-29 [1] CRAN (R 4.0.0)
## utf8 1.1.4 2018-05-24 [1] CRAN (R 4.0.0)
## vctrs 0.3.1 2020-06-05 [1] CRAN (R 4.0.0)
## withr 2.2.0 2020-04-20 [1] CRAN (R 4.0.0)
## xfun 0.14 2020-05-20 [1] CRAN (R 4.0.0)
## xml2 1.3.2 2020-04-23 [1] CRAN (R 4.0.0)
## yaml 2.2.1 2020-02-01 [1] CRAN (R 4.0.0)
##
## [1] /Library/Frameworks/R.framework/Versions/4.0/Resources/library