Reports the optimizer's own verdict plus six checks that a converged
fit can still fail: non-finite standard errors, flat directions,
complete separation in a binomial-type fit, a distributional
parameter whose maximum likelihood is outside its own range,
predictor columns scaled far from one, and variance components on
the boundary of their parameter space (lme4's isSingular(), read
off the estimates rather than the Hessian).
Details
NON-FINITE TRIALS are points where the optimizer found the objective
undefined and stepped back, for example a line search that crosses
below zero on the 1/mu^2 link. They do not make a fit wrong, so they
are reported as a count and do not count against "No convergence
problems detected". A large count on a model with no restricted link
can point to a density that is undefined where it should not be.
A DISTRIBUTIONAL PARAMETER AT THE END OF ITS LINK is one whose
estimate is far out on the link scale AND whose standard error is
larger than the estimate itself. The likelihood was still rising
where the optimizer stopped, so the number reported is the stopping
point and not an estimate. student()'s nu does this on any data
with no heavy tails, because a student-t reaches the gaussian only
as nu goes to infinity: the maximum is never attained, and two
runs of the same model can report degrees of freedom orders of
magnitude apart while agreeing on every coefficient. Refit with
gaussian(), or hold nu somewhere finite with set_prior().
The same parameter runs the other way, down to one, on data whose
tails are heavier than any identified nu can hold. The check
names that too, with a large NEGATIVE estimate and a natural-scale
value of one, and there gaussian() is the wrong answer: the data
is the message, and a prior is the way to hold nu finite.
THE GRADIENT is reported four ways, because the largest component on
its own is not a convergence verdict. max_grad and worst_grad are
that largest component and its parameter. grad_bound_held names the
parameters a bound holds in place, whose gradient points out of the
feasible set and is the KKT condition rather than a failure, and
grad_proj and grad_proj_par are the largest component over the rest
and the parameter it belongs to. On a bounded fit worst_grad and
grad_proj_par are DIFFERENT parameters, and grad_proj_par is the one
the verdict is about. grad_headroom is the log likelihood one exact
Newton step over those parameters would still buy, which is what the
fit warns on; it is NA when grad_proj stayed under
frmtmb_control(grad_tol =), so nothing had to be measured, and also
when the curvature there is unusable.
TWO SCALES, on an autoscaled fit. max_grad and worst_grad are read
off the RAW gradient, because that is what they have always been and
what extension test suites compare to a log likelihood. grad_proj,
grad_proj_par and the warning are in the per-parameter units
frmtmb_control(autoscale = ) judges the fit in, which is where a
badly scaled column's coefficient is not swamped by its own scale. So
on a fit that was standardized internally the two are not comparable:
1001 against 1.685 on one measured example. The printed block says so
when it applies. grad_headroom is invariant to that scaling, because
a Newton decrement is invariant under any reparameterization.
grad_headroom costs a Hessian, and it is measured only when
grad_proj exceeds grad_tol. A fit that already warned carries the
answer on it, so diagnose() is free there; a fit whose cache was
replaced, which is what perturbing the estimates does, pays between
0.13 and 0.45 s on the large latent-variable designs this package was
timed on.
NOT ON AN IMPORTANCE-CORRECTED FIT. frm(importance = ) optimizes a
Monte Carlo estimate, whose gradient carries an O(N^-1/2) error, so no
gradient criterion applies to it at all. grad_proj, grad_proj_par
and grad_headroom are NA there and the printed block says why. That
fit's accuracy report is its effective sample sizes and its
round-to-round move, and its gradient is in fit$importance$grad.
"No convergence problems detected" is still decided there, by
max_grad against frmtmb_control(grad_tol =): an absent verdict is
not a complaint, and withholding the line from every such fit would
report a problem on fits that have none.
A FLAT DIRECTION is an outer parameter the likelihood does not depend
on: zero gradient and an empty Hessian row. It separates the two
causes of NaN standard errors. Parameters that trade off against
each other are over-parameterization, and the model is too big.
Parameters the likelihood is flat in are unidentified AT THIS POINT,
and the remedy is a starting value: a nonlinear term evaluated
outside its own support (a bump whose centre starts far from the
data) is flat in several of its parameters at once. bad_se names
the parameters without a standard error; the others keep theirs (see
check_se in frmtmb_control()), and flat names the cause. The
check is measured by perturbing each candidate and seeing whether the
gradient moves, and runs only when the covariance has already failed.
Examples
set.seed(1)
dd <- data.frame(x = rnorm(100), g = factor(rep(1:10, 10)))
dd$y <- rnorm(100, 1 + 0.5 * dd$x + rnorm(10, 0, 0.8)[dd$g], 1)
fit <- frm(bf(y ~ x + (1 | g)) + gaussian(), data = dd)
diagnose(fit)
#> Optimizer convergence code: 0 (relative convergence (4))
#> Max |gradient|: 1.45e-05 at sigma_(Intercept)
#> Hessian positive definite: TRUE
#> No convergence problems detected
# a random effect the data cannot support collapses to the boundary,
# which is a valid fit but a warning about the model
dd$h <- factor(rep(1:10, each = 10))
fit_s <- frm(bf(y ~ x + (1 | g) + (1 | h)) + gaussian(), data = dd)
d <- diagnose(fit_s, quiet = TRUE)
d$singular
#> NULL
# a predictor scaled far from one slows the optimizer down; the
# remedy is frmtmb_control(autoscale = TRUE)
dd$xbig <- dd$x * 1e5
diagnose(frm(bf(xbig ~ 1) + gaussian(), data = dd), quiet = TRUE)$scale
#> NULL