Add a model comparison criterion to a fit — add

brms's add_criterion() computes loo(), waic(), kfold(), loo_subsample(), bayes_R2(), loo_R2() or the marginal likelihood once and stores it in the fit, so that a later loo() or loo_compare() reads it instead of computing it again. Every one of those criteria is a posterior quantity: an elpd, a pointwise predictive density or an R2 averaged over draws, or a marginal likelihood integrated over the prior. A maximum-likelihood fit has none of them, and frmtmb refuses loo(), waic() and bayes_R2() on a fit for that reason. So add_criterion() on a fit refuses each criterion by name with the same reason. AIC() and BIC() are the maximum-likelihood comparison, and they need nothing stored: they read the log likelihood the fit already holds. On frmtmb.sample::frm_sample() draws, add_criterion() stores the criteria as brms does.

Usage

add_criterion(x, ...)

# S3 method for class 'frmtmb_fit'
add_criterion(
  x,
  criterion,
  model_name = NULL,
  overwrite = FALSE,
  file = NULL,
  force_save = FALSE,
  ...
)

Arguments

x: A frmtmb_fit.
...: Refused: an argument the method does not have is an error naming it.
criterion: brms's criterion names: a subset of "loo", "waic", "kfold", "loo_subsample", "bayes_R2", "loo_R2" and "marglik".
model_name, overwrite, file, force_save: brms's other arguments, accepted so that a ported call reaches the refusal. This method always stops.

Value

This method never returns; it signals an error.

Examples

set.seed(1)
dd <- data.frame(x = rnorm(40))
dd$y <- rnorm(40, 1 + 0.5 * dd$x, 1)
fit <- frm(bf(y ~ x), family = gaussian(), data = dd)
try(add_criterion(fit, "loo"))
#> Error : add_criterion(criterion = "loo") has no meaning on a maximum-likelihood fit: an elpd averages the pointwise likelihood over posterior draws, and a fit has one estimate and no draws. Compare maximum-likelihood fits with AIC() or BIC(), which read the log likelihood the fit already holds, or with anova() for nested ones. For the criterion itself, sample first: add_criterion(frmtmb.sample::frm_sample(fit), "loo")

# the maximum-likelihood comparison
fit0 <- frm(bf(y ~ 1), family = gaussian(), data = dd)
AIC(fit, fit0)
#>      df      AIC
#> fit   3 110.1729
#> fit0  2 125.1745