Leave-one-out expectations of draws

brms's loo_epred(), loo_linpred(), loo_predict() and loo_predictive_interval() on draws from frm_sample(). Each takes the matching prediction (posterior_epred(), posterior_linpred() or posterior_predict()) and summarizes every observation's column under that observation's leave-one-out importance weights, by loo::E_loo(). The weights are the Pareto-smoothed ones of psis(): one frmtmb::log_lik() matrix and one loo::psis() per call, on the same draws as the prediction.

Usage

# S3 method for class 'frmtmb_draws'
loo_epred(
  object,
  type = c("mean", "var", "quantile"),
  probs = 0.5,
  psis_object = NULL,
  resp = NULL,
  ...
)

# S3 method for class 'frmtmb_draws'
loo_linpred(
  object,
  type = c("mean", "var", "quantile"),
  probs = 0.5,
  psis_object = NULL,
  resp = NULL,
  ...
)

# S3 method for class 'frmtmb_draws'
loo_predict(
  object,
  type = c("mean", "var", "quantile"),
  probs = 0.5,
  psis_object = NULL,
  resp = NULL,
  ...
)

# S3 method for class 'frmtmb_draws'
loo_predictive_interval(object, prob = 0.9, psis_object = NULL, ...)

Arguments

object: A frmtmb_draws from frm_sample().
type: The summary of each column: "mean" (the default), "var" or "quantile", as in brms.
probs: The quantiles for type = "quantile".
psis_object: A psis object from loo::psis(), psis() or loo(save_psis = TRUE)$psis_object, for the same draws and rows. Given, it is used as it is and no frmtmb::log_lik() is computed.
resp: For a multivariate model, the responses to summarize, in brms's spelling (ya) or frmtmb's (y_a). NULL is every response.
...: brms's further arguments. ndraws and draw_ids choose the draws for both halves. The arguments of the prediction function (dpar, nlpar, transform, incl_thres, sort) reach it. r_eff and cores reach loo::psis(). moment_match and reloo are refused when TRUE, as frmtmb::loo() refuses them. newdata, re_formula and the new-level arguments are refused unless they hold brms's default value (NULL; allow_new_levels = FALSE; sample_new_levels = "uncertainty"), which asks for nothing (see Leave-one-out, and newdata). Any other name is an error that names it.
prob: For loo_predictive_interval(), the mass of the central interval, one number in (0, 1).

Value

A matrix with one row per observation and one column per summary, named mean, var or q<100 p>. Where the prediction has a third dimension (categories, or the responses of a multivariate model without resp or with several), an array with that dimension third.

Leave-one-out, and newdata

The weight of a draw for observation i comes from that observation's own column of frmtmb::log_lik(), so a leave-one-out expectation exists only for a row the model was fitted on. brms passes newdata to both loo() and the prediction, and so computes the weights on the new rows' own response. Here frmtmb::loo() and frmtmb::log_lik() refuse newdata, and these functions refuse it in the same words, with re_formula and the new-level arguments. For a summary on other rows, use posterior_predict() or posterior_epred() with newdata: on a row the model has not seen, there is no observation to leave out.

Draws

ndraws and draw_ids choose the draws ONCE, and the same draws give the weights and the prediction. brms draws a random subsample for each of the two, so its weights then belong to other draws than the prediction they weight. As in frmtmb::loo() and brms, the relative efficiency is taken on the chain structure when every draw is used, and on one chain for a subset (ndraws or draw_ids). Draws from frm_sample(thin =) keep their chains.

Multivariate models

Without resp, the weights come from the joint frmtmb::log_lik() of the responses, and the result has a third dimension with one layer per response, named as brms names the responses (ya for a column y_a). With one response in resp, the weights come from that response's own log-likelihood, as brms's loo(resp =) computes them, and the result is that response's matrix. With several, the weights come from frmtmb::log_lik() of those responses (their joint density under set_rescor(TRUE), the sum of their densities without it), and the result has one layer per response named, in that order, as in brms.

Examples

# \donttest{
if (requireNamespace("tmbstan", quietly = TRUE) &&
    requireNamespace("rstan", quietly = TRUE) &&
    requireNamespace("loo", quietly = TRUE) &&
    !frmtmb.sample:::tmbstan_build_broken()) {
  set.seed(9)
  dd <- data.frame(x = rnorm(60))
  dd$y <- rnorm(60, 1 + 0.5 * dd$x, 1)
  ds <- frm_sample(bf(y ~ x), family = gaussian(), data = dd,
                   chains = 2, iter = 500, refresh = 0)
  head(loo_epred(ds))
  head(loo_predictive_interval(ds, prob = 0.8))

  # one set of weights, reused for several summaries
  ps <- psis(ds)
  head(loo_predict(ds, type = "var", psis_object = ps))
}
#> frm_sample(): default priors (brms 2.23 defaults; prior = "flat" puts brms's (flat) in every slot)
#>   Intercept          student_t(3, 0.8, 2.5)
#>   sigma              student_t(3, 0, 2.5)  [natural scale]
#>   b                  (flat), as brms leaves slopes
#> Running PSIS to compute weights
#> Running PSIS to compute weights
#>            var
#> [1,] 0.7788997
#> [2,] 0.8078704
#> [3,] 0.8270408
#> [4,] 0.7616302
#> [5,] 0.9756903
#> [6,] 0.8447159
# }