Skip to contents

Bayesian R-squared

Source: R/loo.R
sample-bayes_R2.Rd

The proportion of the outcome's variance the model explains, computed per draw and returned as a posterior distribution. This is the residual-based estimator of Gelman, Goodrich, Gabry and Vehtari (2019), R-squared for Bayesian regression models, The American Statistician 73(3), which brms implements as var(ypred_s) / (var(ypred_s) + var(y - ypred_s)) for each draw s, with both variances taken over the observations. frmtmb computes the same expression on posterior_epred() draws, so the two agree up to Monte Carlo error on the same model.

Usage

# S3 method for class 'frmtmb_draws'
bayes_R2(
  object,
  resp = NULL,
  summary = TRUE,
  robust = FALSE,
  probs = c(0.025, 0.975),
  ...,
  ndraws = NULL,
  newdata = NULL,
  re_formula = arg_unset()
)

Arguments

object

A frmtmb_draws from frm_sample().

resp

For a multivariate model, which responses, in brms's spelling (ya for a column y_a). NULL, the default, is every response, one R2<resp> column each, as in brms.

summary

If TRUE (the default, as in brms), summarize the draws into estimate, error and quantiles; if FALSE, return the ndraws x responses matrix of R-squared draws.

robust

If TRUE, the summary is the median and MAD instead of the mean and SD, as in brms.

probs

Quantiles for the summary.

...

Refused: an argument the method does not have is an error naming it, rather than silently changing nothing.

ndraws

Number of draws to use (default: all). It follows ... so that brms's positional slots keep brms's meaning.

newdata, re_formula

brms's arguments, which brms hands to posterior_epred(): with newdata the R-squared is of its rows, against its own response column, which it must carry, and re_formula chooses the group-level terms of the prediction, as in posterior_epred().

Value

A one-row summary matrix, or the matrix of draws when summary = FALSE.

Details

Because the denominator is a per-draw variance rather than a fixed total sum of squares, the value is bounded in (0, 1) by construction and does not have the classical R-squared's habit of exceeding 1 on posterior draws.

See also

frmtmb::log_lik(), frmtmb::loo()

Examples

# \donttest{
if (requireNamespace("tmbstan", quietly = TRUE) &&
    requireNamespace("rstan", quietly = TRUE) &&
    !frmtmb.sample:::tmbstan_build_broken()) {
  set.seed(9)
  dd <- data.frame(x = rnorm(60), g = factor(rep(1:6, 10)))
  dd$y <- rnorm(60, 1 + 0.5 * dd$x + rnorm(6, 0, 0.5)[dd$g], 1)
  ds <- frm_sample(bf(y ~ x + (1 | g)), family = gaussian(),
                   data = dd, chains = 1, iter = 500, refresh = 0)
  bayes_R2(ds)
  quantile(bayes_R2(ds, summary = FALSE), c(0.1, 0.9))
}
#> frm_sample(): default priors (brms 2.23 defaults; prior = "flat" puts brms's (flat) in every slot)
#>   Intercept          student_t(3, 0.8, 2.5)
#>   sigma              student_t(3, 0, 2.5)  [natural scale]
#>   sd                 student_t(3, 0, 2.5)  [natural sd scale]
#>   b                  (flat), as brms leaves slopes
#> Warning: The largest R-hat is 1.08, indicating chains have not mixed.
#> Running the chains for more iterations may help. See
#> https://mc-stan.org/misc/warnings.html#r-hat
#> Warning: Bulk Effective Samples Size (ESS) is too low, indicating posterior means and medians may be unreliable.
#> Running the chains for more iterations may help. See
#> https://mc-stan.org/misc/warnings.html#bulk-ess
#> Warning: Tail Effective Samples Size (ESS) is too low, indicating posterior variances and tail quantiles may be unreliable.
#> Running the chains for more iterations may help. See
#> https://mc-stan.org/misc/warnings.html#tail-ess
#>       10%       90% 
#> 0.1828278 0.4194416 
# }