Changelog
Source:NEWS.md
frmtmb 0.80.0
Fitting
BREAKING:
frm(quadrature = TRUE)integrates two nested random-effect blocks, such as(1 | school/class). Before, frmtmb refused a model in which a row reads more than one block. The new route integrates the inner block inside each level of the outer block with Gauss-Hermite rules that the Laplace fit centers and scales, and re-anchors the rules at the estimate. Against independent references the log-likelihood error at the default nodes was 9e-11 to 7e-6 on seven designs with standard deviations near 1, and 5e-4 to 6e-4 with an inner standard deviation near 3 or an outer one of 3; on a near-separable response it was 1.7e-2, and the fit warned that its rule did not settle. The Laplace error on the same designs was 0.07 to 1.4. More nodes do not always reduce the error on the hard designs.A nested fit whose rules do not settle at the estimate warns with class
frmtmb_quad_unsettled. Profile and root-search intervals (confint(method = "profile"),"uniroot",profile(),hypothesis(method = "profile"),conditional_effects(band = "profile")) andranef(condVar = TRUE)stop on a nested fit with classfrmtmb_quad_nested_unsupported; Wald and bootstrap intervals work.frmtmb still refuses crossed blocks,
(1 | g) + (1 | h), and says that their integral does not factor. It also refuses three or more blocks, and two blocks on different predictors, such as(1 | g)inmuand insigma, each with its reason.(1 + x || g)is one block of two coefficients and is refused; the message says to write it as(1 | g) + (0 + x | g), which the nested route integrates.New
frmtmb_control(quad_nodes = )sets the nodes per level of the nested route (default 11 outer, 15 inner). Newfrmtmb_control(quad_max_gb = )(default 4) limits its memory: the fit predicts the memory from small probe tapes and measured costs per family, and stops before the Laplace warm start when the prediction is above the limit.-
BREAKING: a covariate far from zero against its spread fits the same model as the covariate centered. With
y ~ (x + 1e6) + (1 | g)and an sd of 1, the optimizer stopped at its start with code 0. On poisson and Gamma the slope was 6 to 10 standard errors off and the fit 20 to 47 log-likelihood units short, on 5 of 5 data sets each, with no warning on 6 of the 10. Bernoulli was 0.9 to 2.6 standard errors off fromx + 1e5, and a gaussian fit without a group term 12 standard errors off atx + 1e6. The optimizer now steps in coordinates in which each ill-conditioned fixed design has orthogonal columns, the ones the standard errors already used. A design with a condition number below 1e3 keeps its fit. On a grid of offsets from 1e3 to 1e8, four families, with and without(1 | g)and 5 data sets each, all 240 fits give the slope, its standard error and the log-likelihood of the centered fit. Atx + 1e8, 11 of the 40 fits warn of “false convergence (8)” at the maximum; none below that warns. The same holds for:- a
cumulative()fit, which was up to 3.6 standard errors off with no warning at5 x + 1e6; - a shifted column in
sigma’s formula, in aziformula, in an nlpar’s formula, besides(),offset(),weights()or a factor, and its interaction with a factor; - a nonlinear body linear in a shifted column,
b0 + b1 * x, which was 7.6 to 9.9 standard errors off, one of two with no warning; - a prior with
lborubon classb, which kept the old fit, 6.3 to 9.4 standard errors off, 4 of 5 with no warning. A bound on the intercept of such a design, or one that centering does not condition, keeps the old fit and warns (classfrmtmb_ill_design); -
profile()andconfint(method = "profile")or"uniroot"on a shifted coefficient, which wereNAwith an unclassed warning; -
frm_allfit()’s bobyqa and NLopt fits.
On two raw polynomials with
(1 | g)whose group sd the optimizer ran to zero, the fit now reaches lme4’s maximum at an sd near 0.005, 1e-4 higher. - a
BREAKING: the frame drops a fixed-effect column by two tests on the design with its columns scaled to the same size. A column whose QR leaves less than 1e-11 of its norm, with every column but the intercept centered, is aliased and is dropped with the message “rank deficient”. It was 1e-7 on the unscaled, uncentered design, as in
lm()and lme4, which dropped the slope ofx + 1e7with an sd of 1, and products and powers of shifted columns (w1:w2,I(w^2)) from an offset of 1e4: the fit was 0.43 to 50 log-likelihood units below the model asked for, with a message only. Such columns are now kept, as brms keeps them. Then, in their order, the columns that the optimizer cannot fit (a centered condition number above 1e10, or columns dependent to 1e-11 without centering) are dropped with a warning of classfrmtmb_design_capped: the data identify them, so the model fitted is not the one written. Products and powers of columns shifted by 1e6 meet this warning. A raw polynomial far from zero,poly(x, 8, raw = TRUE)on [1000, 1010], keeps 5 columns with finite standard errors. A near alias that the old test dropped and the new one keeps, such asx2 = x1 + 1e-9 * noise, now has standard errors near 1e8, asglm()gives. An exact alias is dropped as before.frmtmb_control(autoscale = TRUE)on a column far from zero warned of “false convergence (8)” at the maximum. Fromx + 1e3such a fit is now silent, and its standard errors are those of the centered fit.-
BREAKING: a nonlinear fit from its default start also fits from each sign pattern of the body’s coefficients and returns the highest point. When that point is higher than the first fit by more than 0.01, a message of class
frmtmb_nl_multistartsays so. Before, a default fit could stop at a lower maximum at return code 0 with no warning:b1 * (1 - exp(-b2 * x))undergaussian()stopped at its linear limit, 69 to 75 below the maximum on 5 of 5 data sets, and abernoulli()b1 * exp(b2 * x)stopped at the other sign ofb2.- The patterns vary each coefficient that the body does not read as an added term, so
a + b * xis not affected. With four such coefficients or fewer, every pattern runs; with more, every pattern of the four that move the body most. The order of the terms does not change the result. - A coefficient that moves the body far more or less than 1 per unit starts at the body’s scale:
b2inb1 * exp(b2 * x)withxin the thousands starts near 1e-3. - On 450 fits (9 families, 5 bodies, 2 slopes, 5 seeds), 16 stopped more than 1e-4 short of the best of 8 random starts, 11 of them with no warning; now 0 do. On 335 other fits (15 bodies, 6 families, 5 seeds), 97 were short, 38 with no warning; now 11 are, 5 with no warning. They are Michaelis-Menten bodies under
bernoulli()andbinomial(), a power law undercumulative(), a four-parameter logistic underbernoulli()2.8e-4 short, and fits that warn. On these 785 fits none is lower than before. - The point a pattern reaches can itself be a second maximum. The fit then converges with no warning and is still short: on 1 of the 87 fits that the patterns raised, 1.84 short, where the default start alone stopped 2.29 short with a warning. A
startelsewhere can show it. - The cost is 1.2 to 2.7 times the fit time with two or three such coefficients, and 3.5 times with four, on fits of 300 rows. On fits of 20000 rows it is 1.5 to 2.8 times with two or three, 5.4 times with four, and 47 times with five, where the fit gained 15906 log-likelihood units.
- Give
start, or setoptions(frmtmb.nl_sign_probe = FALSE), to skip it. A located prior does not skip it.
- The patterns vary each coefficient that the body does not read as an added term, so
BREAKING: a nonlinear coefficient that the body reads as an added term is stepped in the units by which it moves the body, where these units are 16 or more times from 1. The curvature of a normal, Student t or logistic prior on it is added; under another prior with a curved density, such as a gamma or lognormal prior, it keeps its old steps.
a * xwithxnear1e-20now reaches the maximum ofy ~ 0 + x; before, it stopped at its start, -299.33 against -272.50, at return code 0.a + b * xwithynear1e6now reaches the maximum ofy ~ x; before, it stopped at -3045.62 against -1658.76. In the sign patterns’ runs and the refit of their winner, a coefficient inside a curve is stepped in its start size where that is 16 or more times from 1, and keeps its old steps under a prior.b1 * (1 - exp(-b2 * x))andb1 * exp(b2 * x)withxnear 1e-6 now fit as withxnear 1 on 40 of 40 data sets.When the refit from the best sign pattern does not stop where that pattern’s run stopped, and raises no warning of its own, frm() warns with class
frmtmb_nl_multistart_unsettled.BREAKING: an exact
gp()in a parameter of a nonlinear body, such asbf(y ~ a + b * z, a ~ gp(x), b ~ 1, nl = TRUE), starts at the maximum of the marginal of the body made linear at its start. Before, the field had no warm start.frm()also descends from the old start and keeps the better fit, so no fit reads lower than before. On 192 test fits of 12 bodies,frm()stopped short on 56, by up to 290 nats, with return code 0 and no warning. Now 8 stop short, by up to 36.5 nats, also with no warning, and none reads below its old value. 5 of the 8 are low-count Poisson fits, where the linear spellingy ~ z + gp(x)stops at the same value (see the Limits of 0.79.0). On a gaussian body additive in the field’s parameter (a + b * z,a + b * z^2,a ~ gp(x) + (1 | g),a ~ gp(x, by = f)) no fit stops short, which closes the gap of a nonlinear body that the Limits of 0.79.0 name, 98.5 nats on a trend plus a wiggle. With astartof your own the field starts as before. These fits take about 2 to 2.2 times as long.BREAKING:
frm()now leaves a ridge that the restart from a saddle did not reach, on a model with two or more nonlinear bodies. The restart moved only the body that was constant; when the restarted fit still warns that it may not have converged,frm()now also moves the other bodies.nlf(mu ~ b1 * exp(b2 * x))besidenlf(disc ~ exp(d1 * d2 * z))undercumulative()stopped at the linear limit ofexp(), at log-likelihood -377.03 with return code 1, where a start reaches -371.54. On 30 such fits, 20 stopped short by up to 5.3 nats, each with a warning. Now none does. These fits take about 1.4 times as long. A model with one body does not change.BREAKING:
frm()fitsmi()on the response beside a latent residual when no other density reads the missing values. Before, it refused this model with a reason that is false for it. Those rows leave the objective, and the latent residual of such a row has no observation. On lognormal the log-likelihood is the exact marginal of the observed rows to 1e-8.frm()still refuses the model when another response reads the missing values (bf(w ~ mi(y))).-
BREAKING: an exact
gp()besidear()on a gaussian response now starts at the maximum of the model’s own marginal. That marginal includes thearcoefficients. Before, the start read a marginal of the response alone.brms’s default
ar()(cov = FALSE) moves the mean of each row by the earlier residuals of its group. So the field integrates out in closed form beside it. This holds forar()of any order, with or withoutgr, and with a freesigmaand nose()orweights(). The start is used only where the objective reads the marginal’s value at the marginal’s maximum, to1e-6.On a step-shaped field, n 160, the fit stopped 25.9 nats short with convergence code 0 and no warning. Five more such rows stopped 0.43 to 26.8 nats short. Now all six reach the best value that other starts found, and two read above it.
On step-shaped data beside
ar(), the maximum now usually has anarcoefficient at or just above 1: a unit root takes the step, and the field is smaller. On the step row above it is 1.008, with sdgp 0.32 against 0.75. This is the likelihood of brms’scov = FALSEform, whosearis unconstrained in brms too.The time of a fit beside
ar()changes. In one process, against the former start: 0.52 times at 500 positions, where the fit also read 7.8 nats higher; 1.2 to 1.3 times at 300 positions; on panels, with rows of many series on shared positions, 1.5 to 1.7 times on 1000 rows of 40 positions and 2 to 5 times on 600 rows of 50 positions, under about 2 seconds in each.-
BREAKING: off the gaussian, where the start’s marginal of the response has a second maximum at another length scale,
frm()also starts from it and keeps the better fit. This was read on the 150 rows of bernoulli, Poisson, binomial, negative binomial, Gamma, lognormal, student,sigma ~andar()fits on which the last release named its limits. Before, 13 stopped short by more than 1e-3 nats, worst 25.9. Now 5 do, worst 11, and no row is lower.The second start ran on 40 of those fits and took 13 percent of the time of all 150. On 72 rows of smooth curves it ran on 25 and took 15 percent. A single fit where it runs and loses takes up to about 2.2 times as long: 14.5 s against 8.4 s on a Poisson fit of 240 rows, and 32.9 s against 15.2 s on a bernoulli fit of 240 rows.
A constant
sigma(bf(y ~ gp(x), sigma = 0.3)) is a known noise in the start of an exactgp(). Before, the marginal read no noise at all: at the fit’s optimum it stood 1.6e12 from the objective, and its maximum was both the start and the value the convergence check reads. Besidear()a constantsigmatakes the working start.The convergence warnings of a fit have their own classes, before
frmtmb_warning:frmtmb_conv_code(“Optimizer did not report convergence”),frmtmb_conv_gradient(“Large maximum absolute gradient”) andfrmtmb_conv_gradient_nonfinite(“The gradient at the reported optimum is not finite”). The messages do not change. Code that readsclass(w)[1]sees the new class;inherits(w, "frmtmb_warning")is still true.
Log-likelihood and priors
- BREAKING: under a prior with a density (
frm(prior = )),logLik()is the log-likelihood of the data at the estimates. It does not include the log prior. The newlogLik_penalized()gives the value that the fit maximized, with the log prior, whichlogLik()gave before. brms’slog_lik()is the data part too; glmmTMB’slogLik()withpriors =is the penalized value. Fory ~ x + (1 | g)with anormal(0, 100)prior on the slope,logLik()was 5.52 below the fit with no prior at the same estimates. That is the log of the prior’s normalizing constant. NowlogLik()is the objective of the fit with no prior at the MAP estimates, to rounding. The split is exact for the Laplace approximation,quadrature = TRUE,importance =andfrmtmb_control(profile = TRUE), because a prior there reads only parameters outside the integral. UnderREML = TRUEa prior on the fixed effects is inside the integral, sologLik()stops with a message there. The log prior term is what the optimizer adds, with the log-Jacobian of each change of scale and no truncation normalizer, so it is not brms’slprior;?logLik_penalizedsays how they differ. - BREAKING: under a prior with a density, the estimates do not maximize the log-likelihood, and a parameter that the prior shrinks is not a full degree of freedom. So
logLik()hasdf = NA, with the reason in attributedf_why, andAIC(),BIC()andextractAIC(), and sostep(), stop with a message when they dispatch on such a fit, asanova()anddrop1()do.AIC(m0, m1)stops when either frmtmb fit has such a prior. A call that reaches stats’s own methods with the df oflogLik(), asAIC(lm_fit, map_fit),BIC(lm_fit, map_fit)orAIC(logLik(map_fit)), getsNAfor that fit. A package that counts the parameters itself is not covered:insight::get_loglikelihood()replaces theNAwith its own count, soperformancecan still give an AIC for a fit with a prior. A prior that only sets bounds adds no density, and these methods work; so does auniform()prior on a coefficient or annlpar, whose log density is a constant inside its bounds. -
summary()of a fit with such a prior printslogLikand the log prior in place of AIC and BIC. Under REML with a prior on the fixed effects it prints the maximized value and says why there is nologLik. It marks the method(MAP)only for a prior with a density, not for a prior of bounds alone or a uniform one. -
anova()of twofrm_multiple()fits stops for the likelihood rules ("D3", and"D2"withuse = "likelihood") under a prior with a density, for the reasonanova()of two fits stops. The Wald rules still work: they are Wald tests at the MAP, consistent with the Wald intervals ofsummary(), and under a prior their nominal size is not guaranteed. - The warning for a discrete log-likelihood above 0 now also checks a fit with a prior.
-
frm_allfit()and the restarts of a nonlinear fit still compare the value that each fit maximized, now throughlogLik_penalized(). Under a prior, their printout names that value the penalized log-likelihood. -
logLik_penalized()on afrm_multiple()result stops with a message, aslogLik()does.
Predictions and their standard errors
BREAKING:
fitted()withallow_new_levels = TRUEat a grouping level the fit did not see is the mean of the expected response over that level’s effects, as brms’sposterior_epred(sample_new_levels = "gaussian")is; brms’s default"uncertainty"is still refused. Before, it read the effects at their mean of 0, which is the median: a poisson(1 | g)at sd 1 read low by a factor of 1.647. On a log link that reads the mean the value is the closed formexp(v / 2)times the median.Est.Errorholds the effects’ spread, at the estimated sd, and theQcolumns are their quantiles.sample_new_levels = "old_levels"andscale = "linear"do not change.BREAKING:
conditional_effects()’sse__is on the displayed scale, as brms’s MAD of its draws is. Before, it was on the link scale: a lognormal mean read 0.027 where the response scale reads 0.045, and a poisson mean 0.045 where it reads 0.065. On a row with a newme()value, a latent or a new level,se__and the band arefitted()’s. Elsewhere the band does not change.BREAKING: a new row that reads two independent draws, a latent residual on
mubeside ame()value on another predictor or twome()calls on two predictors, is integrated over both. Before, the second predictor moved by its regression on the first, and ame()value on another predictor thanmubeside a latent was not read. On ahurdle_poisson()row, the draw’s sd read 0.733 against 0.925, and the mean 2.110 against 2.142.BREAKING: a lognormal whose
sigmareads a draw (me(), a latent or a new level) has an infinite exact mean over it, from the draw’s far tail; so does a lognormal component’ssigma<k>of a mixture, and asigmabody that reads a nonlinear parameter with a draw.fitted()gives the 11-node Gauss-Hermite mean, which reaches 5.19 sds of the draw as a sampler’s draws do, with one warning per call of classfrmtmb_mean_not_finite. Onsigma ~ (1 | g)at sd 0.27 it reads 2.0897, as a mean of 4e6 draws does; 0.79.1 read the median, 2.0038. Past a draw sd of about 0.3 onsigmano mean of draws is stable, and the warning gives how far the rule moves. Before, it gave the value at a far node of the rule: 1.7e162 atsdx = 0.8. Any other mean whose rule does not settle as it grows is read the same way, with a warning of classfrmtmb_mean_unsettled.A nonlinear body has a standard error:
frm_linpred(se.fit = TRUE),fitted()’sEst.Errorand itsQcolumns read the delta method through the body’s taped Jacobian, and its memory grows with the rows, not their square. Before,Est.ErrorwasNAwith no message. Underallow_new_levels = TRUEa row of levels the fit saw answers, and a new level that a body reads has nose.fit, refused with classfrmtmb_nl_new_level;fitted()integrates over that level’s effects.conditional_effects()on a body whose nonlinear parameter readsme()answers, with the band andse__thatfitted()reads on the row. Before, it stopped.BREAKING:
conditional_effects()’s band for a category of a row with ame()value isfitted()’s, the draw’s own quantiles. Category 3 of acumulative()row atsdx = 1.2read 0.196 to 0.305; it now reads 0.090 to 0.254, and the draw’s quantiles at the estimates are 0.098 to 0.247.A draw that moves nothing
fitted()reads, as ame()value on a Gamma’sshape, costs no quadrature nodes. Past four draws that move it, the four that move it most are integrated, with a warning of classfrmtmb_draw_dims.BREAKING: on a latent residual (
ar(),ma(),arma(),cosy()orunstr()on a family with no residual),fitted(),residuals()andfrm_linpred(se.fit = TRUE)give standard errors for the fitted rows at any number of rows. Above 3000 rows they wereNA, with a message. They now read the joint covariance through the sparse Cholesky factor of the joint precision, and each row’s variance through the selected inverse of that factor. On one poisson series withar(cov = TRUE),fitted()took 4.7 s at 2000 rows and 332 s at 8000 rows. It now takes 0.17 s and 0.42 s, and 1.7 s at 32000 rows. Withhu ~ (1 | id)onhurdle_poisson(), orzi ~ (1 | id)onzero_inflated_poisson(), it takes 3.3 s at 32000 rows. The values agree with the dense route to 1.3e-13, relative.BREAKING: beside another group-level block,
fitted()on new rows of a latent residual keeps its standard errors abovefrmtmb.latres_dense_rowsrows. They wereNA, with a message. A new row reads the estimates and the levels of the other block, and no latent.A fit with a latent residual keeps no dense joint covariance after
fitted(). It keeps the sparse factor of its joint precision, which grows with the rows. On one poisson series, a saved fit of 2000 rows grew from 5.1 MB to 36.2 MB afterfitted(), and now grows to 5.7 MB. At 8000 rows it grew to 500 MB with both row limits moved; with the default limits its standard errors wereNAand it stayed at 9.6 MB. It now grows to 12.3 MB, with every standard error given. Beside a group-level block of 50 levels it grows from 11.6 MB to 15.7 MB at 8000 rows, and from 35.6 MB to 51.9 MB at 32000 rows.options(frmtmb.latres_dense_rows = )now limits only a dense block of the joint covariance of a latent residual that a function returns or must form:frm_joint_cov(),frm_lp_basis()on the fitted rows, and the standard errors of the category probabilities of an ordinal family’s fitted rows.frm_curve()andconditional_effects()on new rows are not refused.options(frmtmb.latres_se_rows = )now applies to those category probabilities and to the Laplace check alone. A fit whose covariance of the estimates is held as a factor, which an ill-conditioned design gives, keeps the dense joint covariance and both limits as before.frm_lp_basis()on a fit with a latent residual leaves out ofcoef_posevery column that no row ofAloads: a latent, a level that the rows do not name, and a coefficient whose covariate is 0 on every row. Such a column adds nothing toA V A'. A new row loads no latent.BREAKING: the standard errors of
marginaleffectshold the group-level effects on a fit with a group-level term, a smooth,gp()orhsgp().get_coef()gives the group-level effects at their modes after the coefficients, underfrm_lp_basis()’s names (b.<term>.<level>.<coef>), and thesdgpandlscaleof an exactgp(), whose field reads them.get_vcov()gives their joint covariance with the coefficients and the variance parameters, whichfitted()reads. Before, the delta method held them at their modes, silently wrong on most fits: on 168 fits with(1 | g)or(1 + x | g)on four families,avg_predictions()was more than 25 percent too wide on 77 percent of them (median 2.03 times) andavg_slopes()on 52 percent (median 1.30 times), and the standard error of a level was off on 67 percent. Now the standard errors of the 2016 levels arefitted()’s to within 1e-5, and the averages, slopes and contrasts are within 25 percent of the draws on 164 of the 168 fits; on the other 4 the delta method itself is that far. On a poisson(1 | g)row, 0.4875 where it was 1.4350.avg_slopes()on an exactgp()field reads the field: 0.0308, where it read about 3e-7 on a gaussian field. That is the delta method’s value, asfrm_lp_basis()gives it; on the same data brms’s posterior SD of that average slope is 0.154, because the slope is flat in the length scale near its estimate and steep two standard errors above it. Far from its data an exactgp()is also where marginaleffects’ finite-difference Jacobian parts fromfitted()’s exact one, by up to 21 percent at the points measured, withnumderiv = "fdcenter"no closer; read a level there fromfitted().re_formula = NAtakes the group-level effects out ofget_coef(); a smooth’s coefficients stay. Insidehypotheses()on the fit itself,get_coef()andget_vcov()give the coefficients alone, so"b2 = 0"names the coefficient it named before andjoint = TRUEtests the coefficients and no group-level effect. A function given ashypothesisgets the model and keeps the effects in:hypothesis = function(x) avg_predictions(x)reads 0.143, asavg_predictions()alone does, where holding them fixed read 1.774.vcov =a covariance over the outer parameters, such asvcov(fit, full = TRUE)or a cluster sandwich~ g, is carried into the group-level effects, as the model’s own is. Only the part of their variance that comes from the outer parameters becomes robust; their conditional variance stays model-based."HC0"and"HC1"are refused on a fit with a group-level term, as before. A matrix over the coefficients alone holds the effects at their modes, with a warning of classfrmtmb_interop_se_part.The warning of class
frmtmb_interop_se_partnames what no parameter holds: the draw of a new row’sme()value or latent residual (and, undercov = TRUE, the fitted latent values it continues from), and the group-level effects of a fit withquadrature = TRUEorimportance =. A test of a coefficient no longer gives it.get_predict()gives it once per call, frompredictions(),slopes()andcomparisons()alike, and under the defaultvcovor a covariance the user gives, for rows at a grouping level the fit did not see and for an exactgp()at a position the fit did not see, where the kriging variance is more than 1e-4 of the field’s, and counts the rows of each kind.The time of
marginaleffectsgrows with the number of levels, one prediction per parameter:avg_slopes()on a poisson(1 + x | g)fit took 16.3 s at 500 levels of 40 rows, where it took 0.09 s with the effects held fixed.vcov = FALSEandre_formula = NAavoid it.
Residuals
-
BREAKING:
residuals(type = "osa")reads each row against its family’s own CDF (osa_method = "cdf") on every core family that has one. Only an untruncated, unweightedgaussian()keeps"fullGaussian". With no group-level term a residual is now the randomized quantile residual of the fitted distribution:qnorm(F(y))for a continuous response, and for a countqnorm(u)withuuniform on[F(y - 1), F(y)]. On one design per family, the count families matched it to 1.2e-12 or better and the continuous families to 3e-9 or better. The old default,oneStepGeneric, integrated a spline through a profile of the density, and the results were wrong with no warning:- On counts, the profile read the mass function between the integers.
binomial()read NaN on 2 to 3 of 150 rows.bernoulli()residuals sat near -3 to -4 with or without a group-level term (57 of 150 rows outside their own step with none). Six other count families put 5 to 13 of 150 rows outside their own step, andbeta_binomial()was refused. -
Gamma()read up to 0.086 off,Beta()0.69,von_mises()4.45, andinverse.gaussian()andfrechet()read NaN on 3 rows and up to 7.3 off.tweedie()read NaN on 71 of 150 rows. -
student(),exgaussian(),asym_laplace(),exponential(),weibull(),lognormal(),shifted_lognormal()andhuber()stopped with an error from inside RTMB.huber()has no CDF, and it is now refused with the reason.
A residual is read in the tail its row lies in, so it holds to about +-37 and beyond: a poisson count of 60 about a mean of 3 reads 15.24 where TMB’s own formula read
Inf.With a group-level term, TMB integrates each row by the Laplace approximation. Against the exact one-step predictive the median error was below 0.001 under
(1 | g)and 0.006 to 0.016 under a latentar()on poisson and the zero-inflated and hurdle counts. The largest error was 0.15, on the first row of a group. Over many rows the residuals stay calibrated against N(0, 1). - On counts, the profile read the mass function between the integers.
The uniforms of a count’s residual are TMB’s,
set.seed(123)unlessseed =is passed, so the same fit gives the same residuals.residuals(type = "osa")hands the rows tooneStepPredict()in blocks of aboutsqrt(n), so that the objective carries the CDF of one block at a time. On(1 | g)fits of 500 and 2000 rows the residuals took 0.05 to 0.31 of the old route’s time on poisson, negbinomial and bernoulli: 59 s against 303 s on a poisson fit of 2000 rows, which took 34 minutes with every row at once.residuals(type = "osa")works onbeta_binomial()and on the zero-inflated and hurdle count families:zero_inflated_poisson(),zero_inflated_negbinomial(),zero_inflated_binomial(),zero_inflated_beta_binomial(),hurdle_poisson()andhurdle_negbinomial(). The"cdf"route gives the density the count as data, so a point mass at zero is read as it is in the fit.residuals(type = "osa")on atrunc()count reads each row inside its own window, so bounds that change from row to row work.residuals(type = "osa")on a truncatedgaussian()takes the"cdf"route too. With a group-level term the old route left the later rows’ truncation terms in each row’s integral, and on a test of 96 rows the integral failed on every row, so the call stopped with “$ operator is invalid for atomic vectors”.BREAKING: in
residuals(type = "osa")a case weight is a frequency, as in the likelihood and in brms: a row of weightwgets the residual of the last ofwcopies, and later rows are conditioned on allw. A weighted poisson fit with(1 | g)gave the residuals of the fit with the rows written out to 8e-7. A weightedgaussian()now takes the"cdf"route, where it used to read a weight as a precision; with(1 | g)it was 0.0018 from the rows written out, which keep the exact"fullGaussian", and that is the Laplace error of the"cdf"route. A row of weight 0 getsNA, and so does a row of weight between 0 and 1, with a warning. Before, a weight reshaped a count’s own distribution: at weights of 3, 42 of 150 poisson rows fell outside their own step.BREAKING:
residuals(type = "osa", osa_method = "oneStepGeneric")on a count needsdiscreteSupport, the gaussian methods are refused on a count, andtweedie()is refused by name. Before, the first read NaN with no warning, the others gave a gaussian approximation of the count, andtweedie()read NaN. A residual thatoneStepPredict()cannot form gives a warning. A fit withquadrature = TRUEis refused with the reason: its objective has integrated the random effects already.bernoulli()andnbinom1()supply a CDF (cdf_log). Neither takescens()ortrunc(), which are still refused, now with “is not supported for this family”.
Performance
- The standard errors of an exact
gp()fit of a gaussian response come from the gradient of the model’s marginal, where that is the objective and its covariance is well conditioned. The gradient of the fit’s own objective solves for the field each time. At 500 positionsfrm()andsummary()together took 34.2 s against 53.0 s, and the Hessian alone 2.1 s against 37.2 s; over three runs at 500 positions and at 2000 rows the Hessian’s time fell by a factor of 14 to 18. At 100 positions the factor is 3.5, and the fit as a whole does not change measurably. On 377 test fits the standard errors moved by at most 2.2e-8 of their value, and no standard error changed between a number andNaN. Where the noise is small against the field, the marginal loses digits, so these fits keep the old route.
Package code and continuous integration
The package code does not use the global assignment operator
<<-. rOpenSci’s pkgcheck fails a package that does. A closure that keeps state between calls now writes to an environment that it creates, and a function that collects results returns them. The results do not change. The testtest-no-superassign.Rfails if<<-comes back in core. It also fails if<<-comes back in an extension, but only when the tests run from the source repository.The test “new data are coded from the fitted minimum” compares a prediction on 4 rows with a prediction on 4 rows. On Ubuntu 26.04 with OpenBLAS 0.3.32, one value of a prediction on 3 rows was 1 ulp from the same value on 4 rows, and the test failed. The 3-row prediction must now agree to 64 ulps. A code from the minimum of the new data moves a value by a full category, so the test still finds that defect.
The test “the nl body reaches the likelihood” fits the negative binomial families and
student()to data with a maximum inside the parameter space. On Poisson or normal data, the shape ornugoes to infinity, so the point where the optimizer stops, and its warning, changed with the platform. A gradient warning on Ubuntu was not in the allowed list.Continuous integration: the check for a skip on a missing Suggests package runs also when the check or the coverage run fails, so one run shows all failures. The weekly
ubuntu-nextworkflow has this check too, and has 150 minutes for its Stan tiers, which compile every program from an empty cache. Its record of the numerical libraries also tries to show the CPU, the OpenBLAS kernel and the OpenBLAS package. Thebrms-tiersjob has 180 minutes and its identity tier 120, because an empty cache and a slow package mirror together can take more than 90.
Limits
The limit the last release named beside
ar()on a step-shaped field is closed. The one onstudent()there is not: 11 and 6.07 nats short on two rows. The Student-t density is not log-concave in the field, so the inner solve can find more than one mode, and the log-likelihood at one point depends on where that solve starts: 207.37 from one inner start and 204.77 from another. A second start does not fix that. Two more rows stay short, 1.59 on Poisson and 1.45 onsigma ~ z, as they were.A body that multiplies the field’s parameter is reached only in part.
a * exp(b * z)witha ~ gp(x)stops 36.5 nats short on 1 of 12 test fits, anda * exp(b * z)withb ~ gp(x)5.2 nats on 1 of 12, with return code 0 and no warning, as before. Astartforthetais how to check such a fit.A column far from zero against its spread still costs these fits, each with a warning. Beside a
mo()term the fit reaches the maximum, but its curvature is not usable and its standard errors are NaN. A category-specificcs()term on it, or a random slope on it,(1 + x | g), stops short of the maximum. Center the column.Under an ordinal family, whose thresholds take the intercept’s place, products and powers of columns shifted by 1e6 (
w1 * w2,w1 + I(w1^2)) stop 8.6 to 28.6 log-likelihood units short of the model on the centered columns, each fit with a convergence warning. Center the columns.quadrature = TRUEandimportance =run their own objectives in natural coordinates: on bernoullix + 1e6a quadrature fit was 1.2e-5 short with no warning, and importance fits warned “false convergence (8)” at the maximum.A nonlinear body that a bound holds runs in natural coordinates, with no warning.
A nonlinear body that is curved in a shifted covariate,
b0 * exp(b1 * (z + m)), still stops short fromm = 100, by up to 39.5 log-likelihood units and 8.7 standard errors, each fit with “false convergence (8)” and a gradient warning.A nonlinear body with an alias inside it, such as
b0 + b1 * x + (b2 + b3) * t, has a singular Jacobian and gets no second descent: on a shiftedxtwo of three fits stopped 20 and 36 log-likelihood units short, with a gradient warning only.The rounds of nested quadrature also run in natural coordinates. On a bernoulli
(1 | school/class)fit withx + 1e4orx + 1e6, the fit was 0.0007 to 0.039 below the centered fit and its slope up to 0.06 standard errors off, with a warning that the rule did not settle and one of false convergence. Center the column.
frmtmb 0.79.1
residuals(type = "osa")onbeta_binomial()always stops with a message. RTMBdist 1.1.0 refused this route. RTMBdist 1.2.0 gives a value, and on a test of 150 rows 21 values were NaN with no warning. The refusal now does not change with the version of RTMBdist.The test “a slope near separation is never silent at few draws” accepts a third result: the optimizer stops and warns. On 61 data seeds at 8 draws, 2 fits stopped with the refusal, 55 were capped and warned, and 4 stopped in the optimizer and warned. None was silent. The result differs by platform.
frmtmb 0.79.0
The main changes:
- BREAKING: the exact
gp()is non-centered, brms’szgp, at a nugget of1e-12 * sdgp^2, and starts at the maximum of the model’s marginal. - BREAKING: an autocorrelation term on a family with no residual
y - muis brms’s latent residual, asar(cov = TRUE)and more. - BREAKING: a smooth’s standard deviation has brms’s class
sds. - New: a nonlinear body under
cumulative(),sratio(),cratio()andacat(), and on aGamma()orinverse.gaussian()default link. -
fitted()on a new row ofme()or of a latent residual is the mean over that draw, andEst.Errorholds the draw. -
frm(importance =)reads the Laplace fit’s own scale, in fewer rounds. - Update the extensions together with frmtmb. frmtmb.sample 0.26.0, frmtmb.learn 0.7.4 and frmtmb.latent 0.7.2 need frmtmb 0.79.0.
Breaking changes
The exact
gp()is non-centered:bof the block is z, a standard normal, brms’szgp. The field issdgp * L(lscale) * z, withLthe Cholesky factor of the kernel plus a nugget of1e-12on the correlation. The field was centered at a nugget of1e-6. Its white noise of variance1e-6 * sdgp^2came offsigma^2. It instead movedsdgpand the length scale when positions repeat,sigmais known or the family has none. Every fit that relied on it moves. On a curvex^2over x from 0 to 10 withsigma0.3,frm()readsigma6e-5 and now reads 0.35, the marginal’s maximum. Onsin(2 * x)with noise 1e-3 it readsigma1.3e-7 and now reads 1.1e-3, 1.8 nats higher. At a knownse(3e-4)it was 29 nats short. Onexp(0.8 * x)withsigma0.3 the fit was 72 nats short and now reaches the maximum, to 0.0005 and 0.0015 nats on two seeds. The objective rounds at 1e-3, and a warning says so.The nugget is relative to
sdgp. So the model ofy * cis the model ofyscaled. A fit of it is the fit ofyscaled to the optimizer’s tolerance. This holds on 5 designs at noise 1e-3 and above,cfrom 1e-3 to 1e6. The log-likelihood is within 3e-7 nats,sigmawithin 3e-5,sdgpwithin 3e-4 and standard errors at new positions within 4e-5. At noise 1e-4, within 3e-4 nats, 4e-3 insdgpand 3e-4 in the length scale. The same holds forx * c. brms’s nugget is an absolute1e-12: aty * 1000its kernel does not factor at the estimates and its chains do not mix (Rhat 3.77), and aty / 1000it floorssigma. Atsdgpnear 1 the two are one number.fit$estimates$b,fit$obj, the joint precision andstart = list(b = )read z in an exactgp()block, over a farthest-first order of the positions.ranef(),coef()andgetME(fit, "b")still read the field. Read the field off those. The kernel is factored in that order because rounding moves the field least there: at 15 positions 6e-11, against 2e-5 over sorted positions. A fit reads the same in either order, and a chain at truesigma1e-3 mixes better in this one.An exact
gp()starts where the model’s marginal, which integrates the field out, reads its maximum. The fit stays on the z tape: only where the descent begins changes. On a gaussian response with a constantsigmathat is the model’s own marginal. It is read from a coarse grid over the length scale and the noise, on the distinct rows of the random design. It starts from the best cell of each of 6 length scales and the best 3 cells overall. Another family, a non-identity link,sigma ~,ar(),mi(),me(),mo()or several responses read the same marginal of the response on its link’s scale. This start is forsdgpand the length scale. Athetainstartreplaces this, and abetaorbetadreplaces only itself.What that reaches, on rows no test had seen. A descent from the middle of the range stopped 417 nats short at
sigma1.19 with no warning. This was on a curve with a trend and a wiggle,2 * sin(3 * x) + x, n 80, noise 1e-3. 0.78.0 reached it there and stopped short elsewhere. Gaussian, on 744 rows of fresh curves: 1 short by more than 1e-3 nats, worst 0.56, conv 0 and no warning. By comparison, 0.78.0 read 40 short and 62 short on the two tables of 192 it was read on.0 + gp(x),start = list(beta = ),mi(),me()andmo()read 0 short on their row. Negative binomial, Gamma, lognormal, student,sigma ~ x, a log link,ar()and two responses read 0 of 48 short. Against this, it was 32 of 48 on 0.78.0, worst 272.No start is certain: a marginal can have more than one maximum.
frm_curve_deriv()on a fit with an exactgp()takes a default step of5e-4of the grid’s range at order 2, where it was1e-4. At the smaller nugget a second difference meets the rounding of the kriging row: at1e-4a standard error read up to 20 percent off.class = "sds"is brms’s class for the standard deviation of a smooth’s penalized part, onfrm()and onfrm_sample().coefnames the term as written ("s(x)","t2(x, z)","s(x, by = f)"). A term with several standard deviations (at2(), a by-smooth) takes the density on each of them.default_prior(),validate_prior()andprior_summary()list a class row withlb = 0and one row per term, as brms does. Class"sd"no longer reaches a smooth, in brms’s words:class = "sd", group = "s(x)"and a class"sd"prior on a model with no random effect are refused, andsdbeside(1 | g)leaves the smooth alone. brms’s log density andfrm()’s objective are equal up to one offset, the same at every point tried. The offset is the brms normalizer of a half density and the map’s Jacobian at the first point.On
frm(), a missing response value that no density reads leaves the objective. It integrates to 1, so the fit is the complete-case one, exactly, andlogLik(),AIC()andBIC()with it. brms’s posterior for every other parameter is that one too. The row stays in the frame and predicts;fit$estimates$misshas no entry for it. This changes every fit of agaussianorstudentmi()response no predictor reads that worked before: a gaussian’s by nothing, astudent’s by a Laplace error. The error movednuby up to 33 standard errors on 200 rows with 60 missing and fitted a truenu = 6as a gaussian. brms’s log_prob on the observed rows equalslogLik()to 4e-13 on 13 families.A gaussian response fitted with
importance =now returns the exact optimum at any draw count, to about 1e-9 in log-likelihood. It used to move away from it by up to 0.09.The stop that said “ROSE” and blamed an unidentified covariance is gone, because the cause was the frozen proposal: the design it named now settles on the exact maximum. A fit that does come out worse than the correction at the Laplace estimates now stops with a message. The message gives the worst group’s effective sample size. It also gives a larger draw count at which the same fit settled with every group above
importance_ess, or names the counts it tried. It tries up to four doublings, up to 8192 draws and 2e6 rows times draws. This is because a tape holds about 1.1 KB per row and draw. So the stop costs more than the fit: 15 times the fit and 3.3 GB at its worst, measured on 244 rows from 512 draws. A larger count that cannot run becomes part of the message.The warning that a correction “moved by the same amount” in every round is gone, with
importance_rounds’s text about it. No fit reaches it now: a collapsed variance component settles where the Laplace fit put it.beta_binomial()andzero_inflated_beta_binomial()have a new density form. A fit on data with no overdispersion changes. The old form added rawlgamma()values of the shapes, and those differences lose their digits asphigrows. The true density is flat there, the binomial’s, so the optimizer went on to where the rounding pointed. On one data set the true value is -548.98 at every largephi. The objective read -66752 at log phi 39 and +327680 at 45.frm()returned +94208 with convergence code 0. On another, -541.09 with nothing to mark it, 0.15 above the binomial limit. These fits now reach the binomial limit with code 0, on the five data sets of the test. A fit with overdispersion keeps its log likelihood to 11 digits. The censored and truncated forms use the same mass function. The time of a fit at 3e4 rows is 0.7 times the old time at 5 trials per row. The ratio is 0.9 at 10, 1.9 at 30 and 3.1 at 200.nl = TRUEoncumulative(),sratio(),cratio(),acat()andhurdle_cumulative()now fits the body. Since nonlinear formulas reached the ordinal families, the rule that thresholds replace the intercept also took the intercept of a nonlinear parameter. Sob1 ~ 1leftb1 = 0, andfrm()andfrm_sample()fitted the thresholds alone without a word: on a test data set the log likelihood was -267.12, the intercept-only fit. By contrast, it was -255.52 for the same predictor written linearly and for the maximum of brms 2.23.0.b1 ~ 1 + zlost its intercept in the same way. The log density now equals brms 2.23.0’s Stan program at three points, with and withoutthres()anddisc. Alsofitted(),predict(),simulate(),conditional_effects()and the prior table read the body and list the thresholds. Onhurdle_cumulative()that holds withoutdisc: with it, brms 2.23.0’s program stops on every top-category row, and frm() fits a model brms cannot.A body that is constant across the rows at a stationary point is no longer returned as a fit.
b1 * exp(b2 * x)at b1 = b2 = 0 is such a point on every ordinal family.a + b1 * b2 * xis such a point on any family. The optimizer stopped there with convergence code 0 and “the data do not determine them”, at the log likelihood of the model without the body: -413.56 on one data set, against -371.76 for brms and for astart.frm()now restarts from moved coefficients of every such body and keeps a higher point. A message gives both values and the optimizer’s return code. It looks at the kept fit again, so a second body does not stay there: asigmabody read 16.8 nats short and amu2body 65.7. If no restart leaves a body, a warning of classfrmtmb_nl_saddlenames it. The warning says that the data may simply have no effect there, which is correct for a slope of exactly 0. It also says that astartcan test it. A fit from the default start that runs along a flat direction, as the standard errors report it, gets the same restarts:acat()on that data set stopped at -378.10, against -372.23. A point that is a true maximum along the body’s coefficients is left alone, with a message of classfrmtmb_nl_local. Every fit that a restart does not beat comes back digit for digit. The restarts do not read the session’s random numbers.optCtrl = list(iter.max = 0)still reads the objective at the start.A fit whose responses are all discrete and whose log likelihood is above 0 now warns, with class
frmtmb_discrete_loglik. A probability mass is at most 1, so that value marks a density that lost its digits or a custom family that dropped a constant. The check is one-sided, and a value below 0 can also be wrong. It does not apply under a prior, wherelogLik()holds the log prior, or under REML.An ordinal fit with a
discbody no longer warns that the family’spost$fit_checkhook failed. The check of an intercept indiscreads a design, which a body does not have. On a body, the standard errors report a constant term that the likelihood cannot place.frm_compat()gives measured statuses fornlby family, in place of a rule that saidrefusedfor every family butgaussian. All core families are “works” exceptasym_laplace(“conditional”: the optimizer can stop at a kink of the density, on the body and on the linear spelling alike). A family no row measures, such as one from a package loaded later, is “untested”.tweedieandnbinom1are “works” and say there is no brms reference.thres(),disc,equidistant,sum_to_zero,mixtureandmvbfbynlare “works”, andcs_pred()bynlis “refused”.ar(),ma(),arma(),cosy(),unstr()andfcor()bynlare “conditional”, where they were “refused”: they work on gaussian and student, and a family whose term would be a latent residual stays refused.frm_linpred(type = "link")with nodparreturns every predictor of the model as a matrix with one column per predictor,eta1,eta2, and so on. This matchesposterior_linpred()of brms 2.23.0, which has one layer per predictor at each draw. An ordinal family with acs()term has one per threshold,eta + cs_k. A family with several location predictors (categorical(),mixture(), the simplex families) has one per predictor. Before, the method returned the sharedetaor the first predictor without a word, wherefitted(scale = "linear")already matched brms.dpar = "mu"anddpar = "mu1"return one predictor.se.fithas the matrix’s shape.frm_curve(),frm_curve_deriv(),frm_curve_feature()andfrm_lp_basis()read the same layers. A curve holds every grid row once per predictor, with the predictor in a new column.eta. The rows are in the orderas.numeric()of brms’s layers at one draw. The band and the covariance are over all of them. Undercs(), the standard error and the derivative read the term’s coefficients: on asratio()fit the derivative in thecs()variable isbcs_z[k]with its own standard error. Before, it was 0 with a standard error of 0.frm_lp_basis()returns the stacked rows and a new fieldlayers.fitted(dpar = "mu", scale = "linear")of acs()fit returns the shared part, as brms 2.23.0 does. Before, it returned one layer per threshold.frm_linpred(dpar = "mu", type = "response"),fitted(dpar = "mu"),conditional_effects(dpar = "mu")andfrm_curve(dpar = "mu", transform = TRUE)of an ordinal family apply the family’s link inverse, asposterior_epred(dpar = "mu")of brms 2.23.0 does:plogis(mu)underlogit. Before, they returnedmuunchanged, because the location’s own link slot is the identity. frmtmb.sample’sposterior_epred(dpar = "mu")andposterior_linpred(transform = TRUE)read the same.emmeans()of acs()fit with nodparaverages the thresholds’ predictors, and~ x | rep.measgives each, as brms does. Before, it read the shared part, whichdpar = "mu"still reads. At the estimate of anacat()fit withcs(f)the means differed from brms’s by 0.083.frm_curve()under acov = FALSEautocorrelation term refuses with its reason, that the mean reads the residual of the row before it. Before, it refused with a message about a misread covariance.A new row of a
me()term has no latent value, and brms 2.23.0 draws one fromnormal(x, sdx).predict(),simulate()andconditional_effects(method = "predict")on new data do the same, per replicate, per row or per level underme(gr = ), also underpropagate_error = FALSE. This is because the draw is measurement noise in the grid. Before, they readx, and the interval was too narrow: at a coefficient of 1.01 andsdx = 0.8, the predictive sd was 0.55 of brms’s. New data must holdsdx, with positive values and one value perme(gr = )level. A level the fit did not see needsallow_new_levels = TRUE, as in brms.cormeis not read, as in brms. A row with noxorsdxreadsNAwith a warning of classfrmtmb_me_missing, as a plain predictor’s row readsNA. brms readsNaNand warns. Sofitted(fit, newdata = )on the fit’s own data answers when the fit dropped such rows.fitted()on such a row is the mean of the expected response over the draw, by Gauss-Hermite on the response scale.Est.Errorholds the draw’s variance beside the coefficients’. On a log link the mean isexp((b sdx)^2 / 2)times the value atx: brms’s mean is 1.41 to 1.76 times that value atsdx = 1.2, and now 0.98 to 1.02 timesfitted(). On lognormal, zero-inflated and hurdle families,Est.Errorwas a constant insdxand now reads 5.22 where brms’s sd is 5.21.frm_linpred()stays at the observedx, which ontype = "link"is exactly the mean. Itsse.fit,frm_curve(),conditional_effects()andemmeans()add the draw’s variance, so their standard errors are not coefficient uncertainty alone, as for a new grouping level.frm_lp_basis()reports it inextra_var, exactly on a product of calls.me(x, sx):me(z, sz)now holds(b sx sz)^2.fitted()’sQcolumns are the interval on the link’s scale, mapped through its inverse, so they stay in the support of the expected response: 0.51 and 17.3 on a poisson mean atsdx = 1.2, where brms’s quantiles are 0.50 and 17.7 and the Wald interval read -2.26 and 8.24. Before, the interval was symmetric about the estimate on every link. On an identity link it still is. On ame()row they are the draw’s quantiles. Over 200 simulated data sets atsdx = 1they cover the mean at the drawn latent on 95.5 and 98 percent. By contrast, the old reading covered it on 88 and 91 percent.One
me()call is one latent value per row and draw, whichever response or dpar reads it. This diverges from brms 2.23.0, which draws one per predictor on new data, against its own Stan program, which has oneXme_1. At the training rows of a two-response fit, the residuals correlate at 0.734, one latent predicts 0.739 and brms’s replicates give -0.001. The 95 percent width ofy1 - y2atsdx = 2is 3.18 with one latent and 15.08 with two. Marginals agree either way for two responses. Onmuandsigmareading one call, the predictive skew reads 0.33, 0.36, 0.63, where brms reads 0.03, -0.02, -0.14. A mean that is a joint function of the predictors moves too: acategorical()probability reads 0.3336, where brms reads 0.3106. A mixture and amuwith azithat read one call are the same.frm_curve(contrast = )reads one draw of ame()value for two rows at onexand onesdx, toall.equal()’s tolerance, so the draw cancels: a contrast inzhas the standard error ofz’s coefficient, 0.051. It answers for one unit with one unknownx.emmeans()contrasts answer for two new rows, each with its own measurement error, as brms’s draws do, so they carry both draws: 0.97 on that coefficient, against brms’s HPD width of 3.77. For the effect ofz, readfrm_curve(contrast = )or the coefficient. For the effect ofx, readfrm_curve_deriv().-
marginaleffects’s defaultvcovgives a warning of classfrmtmb_interop_se_partwhere the standard error of a level frompredictions()cannot hold a variance. This is because the delta method has only the coefficients inget_coef().frm_linpred(se.fit = TRUE)andfitted()carry each of these:- a
me()term: 0.07 against 1.11. A test of a coefficient is right, and so is a slope or contrast on an identity link, -0.491 with a standard error of 0.0485. On a log link a slope in another term reads 0.067 against 0.312. - a smooth,
gp()orhsgp()term: 0.31 on every row of agp(), against 0.087 to 0.695. Ons(x)it reads 0.70, 0.12, 1.07, 1.59 against 0.088, 0.068, 0.55, 1.32. - group-level effects with
re_formulakeeping them: 0.209 against 0.104 on a gaussian(1 | g). Underre_formula = NAthe two agree, and no warning comes. Over 168 fits on four families with(1 | g)or(1 + x | g),avg_predictions()was too wide by more than 25 percent on 77 percent of them. The median was 2.03 times.avg_slopes()was too wide on 52 percent, with median 1.30 times. A gaussian with no group-level slope reads 0.86 to 1.15.
- a
fitted()’sQcolumns of a category probability are a Wald interval on its logit, asconditional_effects()builds it. Before, they were a Wald interval on the probability, clamped to 0 and 1. On a newme()row they are the draw’s own quantiles. On acumulative()row atsdx = 1.2, category 1 read 0 to 0.419 and now reads 0.048 to 0.464. Category 3 reads 0.121 to 0.251. For comparison, brms’s quantiles are 0.048 to 0.469 and 0.135 to 0.244.fitted()’sQcolumns on amixture_mvn()fit are a Wald interval on each mean. Before, they were held between 0 and 1, as for a probability, so they could leave the estimate.conditional_effects()’sse__on a lognormal or hurdle fit moved from the response scale to the link scale, with nome()term too. This matches the help and a poisson fit. It diverges from brms 2.23.0, whosese__is on the response scale: 0.068 against 0.519, as a poisson fit reads 0.102 against 0.809.fitted(scale = "linear")of a distributional parameter with no predictor reads its link scale. This is not new, and it diverges from brms 2.23.0, which returns the stored natural value: 2.01 against 7.48 withshapeunmodeled, and 2.01 in both withshape ~ 1, the same model. On another fit, 0.776 against 2.17.frm_curve_feature()with norespon a multivariate fit asks forresp. Before, it stopped with an internal error.insight::get_predicted()on a multivariate fit returnsRow,Response, the columns ofdataandPredicted, as for brms. Before, it returned one unlabeled number per row and response.frm_linpred()with norespon a multivariate fit returns every response, one column each, asposterior_linpred()stacks them.frm_lp_basis()andfrm_curve()refuse it and ask forresp. Before, the three read the first response without a word.A univariate fit does not read
resp, as in brms:fitted(),predict(),residuals(),frm_linpred(),frm_lp_basis(),frm_curve(),frm_curve_deriv(),frm_curve_feature(),emmeans(),get_predict(),conditional_effects()andpp_mixture()give the same answer for an empty vector. They give the same answer for a name given twice and an unknown name. Before, the first two gave internal errors and the third a refusal.conditional_effects()of a multivariate fit gives one panel set per response withoutresp, and takes several, each named<resp>.<resp>_<effect>, in brms’s order: the effects outside, over what any response reads, and the responses inside, so a response that does not read an effect has a flat panel. A panel of one named response has that name too. Before, it gave the first response alone, named<effect>, and failed on several. A response name given twice is refused here and inpp_mixture(), where brms repeats the panel. This is because brms answers that input four different ways across its methods.conditional_effects()of ame(gr = )fit holds the group atNA, as brms does, so each grid row is a level of its own.frm_curve_deriv()andfrm_curve_feature()in that noisy variable refuse with the reason, that a level holds one value.predict(newdata = )answers underar(),ma(),arma(),cosy()andunstr()withcov = TRUE, through the drawsimulate(newdata = )takes, as brms does. Before, it refused.conditional_effects()on a nonlinear body whose parameter readsme()now stops, with “frm_lp_basis(): the nonlinear body reaches ‘a’”. It answered on 0.78.0. Agp()in such a parameter stops on both, and a smooth answers on both.
New features
gp_nugget_message(), in the sampling API: the new warning’s text, orNULL.gp_tape_scope()andgp_tape_order(), in the sampling API: the tape settings and the parameter order a frame with an exactgp()needs while its tape is built and read.mm()takes an|ID|key,(x | p | mm(g1, g2))ormm(g1, g2, id = "p"). Terms with one key over the same members merge into one block over the pooled levels, in any predictor or response. They take one set of weights, compared by value, soweights = Wbesideweights = cbind(w1, w2)of the same numbers fits: brms reads the first term’s for every term, so different weights are refused rather than picked.mm(g1, g2, cov = A)is a known covariance over the pooled levels, with several columns too, and(1 + x || mm(g1, g2, cov = A))is one block per column, as overgr(cov = A). brms’s log_prob equals frmtmb’s joint density to 4e-9 and 6e-14.gr(cov = A)andmm(cov = A)refuse a matrix that is not symmetric or not positive definite, in brms’s words. A wrong triangle changed the fit by which one the density read, and a matrix that is not positive definite stopped on a NaN gradient.mi()on a response takes every family brms takes it for:skew_normal,exgaussian,gen_extreme_value,von_mises,beta,Gamma,weibull,exponential,frechet,inverse.gaussian,lognormal,asym_laplace, and amixture(), besidegaussianandstudent. Every other family is refused in brms’s words, and a response with no observed value by name.A missing value that a density in the objective reads, a predictor’s
mi()ormi(sd), stays latent onfrm(). A Laplace approximation integrates it out on the chart that holds it in the family’s support: log on the half line, logit on the unit interval, brms’s window(-pi, pi)forvon_mises. A reader whose own density has left does not read:z | mi() ~ mi(y)withzmissing whereyis takes both rows out, along a chain and a fork. A reader’s weight of 0 does not read either. Each of these equals deleting the rows by hand, to 5e-9. A reader’sfitted()at such a row reads the value’s mean. brms’s log_prob equals the joint density to at most 2.3e-8.frm()warns, with classfrmtmb_mi_laplace_error, when that approximation moves an estimate by 0.5 standard errors or more. Three inner Hessians over every read value give Tierney-Kadane’s term, a gate that costs 0.1 percent of a fit. At a term of at most 1 nothing else runs. Past the gate,integrate()gives the move. It runs over 10 read values at the estimate and at one standard error above it, per parameter of the read response. Over 144 fits against an exact maximum it warned on 75 of 76 shifts of 1 standard error or more. It warned on 3 of 39 below 0.5. The largest silent shift was 1.01. The warning names the parameter, the size and the remedy:frm_sample(), or deleting the rows if the reader carries little about the value.frmtmb_control(check_mi = )sets what the check does. On agaussianvalue that onlygaussianmeans read, the approximation is exact and the check does not run. Past the gate the check costs about 37 percent of the time offrm()on such a fit. The approximation can move an estimate. The largest shift from an exact maximum, in its standard errors, on 200 rows: 2.74exgaussianwith 60 read by a slope of 0.05, 2.14studentwith the same, 0.98betawith 100 read by a slope of 0.7. 0.96Gammawith the same, 0.17exponentialwith the same, 0.14exgaussianwith 60 read by a slope of 0.7, 0gaussian. A reader that carries little leaves the row nearly unread, with an unread row’s error:exgaussian‘s parameters sit 2.51 to 3.36 standard errors from the observed rows’ at slopes up to 0.2 with 60 read, and 0.47 at 0.7. Per value, againstintegrate(), on 80 rows with 8 missing and a gaussian reader: 0gaussian, -0.114Gamma, -0.098exponential, -0.078weibull, +0.008frechet, -0.125beta, -0.079exgaussian, -0.076von_mises, -0.037lognormal, -0.023skew_normal, +0.021inverse.gaussian, +0.008gen_extreme_value.von_misesloses -1 to -4.2 per value at a mean near the window’s end, where the value has two modes. Estimates are off by up to 17 standard errors there. Sofrm()warns by name, with no integral, when 10 percent or more of read rows havepi - abs(mu) < 2 / sqrt(kappa). At mean 0 it loses -0.068 per value. Its largest shift over 13 replicates at a mean near 0 is 0.74.frm_sample()has none of this.asym_laplaceand amixture()are refused for such a value onfrm(): the first has no curvature at its kink, and the second has one mode per component.frm_sample()samples both.frmtmb_family(mi_chart = )declares the chart of a family’s missing value ("log","logit","circle","identity","no_mode"). A family with a bounded support must set one.se()onskew_normal, andmi()withse()on one response, as brms writes them. brms’s log_prob equals the joint density to 1e-12.cens()on amixture()whose components have a CDF is the mixture of the components’ tails, as brms writes it. brms’s log_prob equals the density to 6e-14. A group-levelmixture(groups = )refuses it.gr(g, id = "p", cor = FALSE)is accepted when every term under the key sayscor = FALSE: nothing is correlated, so the key does nothing, as in brms. Terms under one key that disagree are refused in both orders, overgr()and overmm(): brms reads the first term’s flag, so swapping them changes its model.id = "p"withcor = FALSEover another group is refused in brms’s words.importanceandvcov(cluster = )run on a fit whose missing rows have all left the objective, and read the observed rows. A value a density reads is still refused: it has no group and no cluster.These stay refused where brms 2.23.0 builds the model, for these reasons.
mi()withcens()when a value is latent: brms’s program takes the density at the data, where a missing value is the placeholderInf. It never reads the latent value, so there is no model to match. With nothing missingmi()takescens()andtrunc().trunc()on amixture(): brms truncates each component and then mixes them, and a CDF of the mixture would be a different model.gr(g, by = , cov = )andmm(by = , cov = ): brms scales each level by the covariance of its by-level and then correlates the levels throughA. This is neither block-diagonal over the by-levels nor a Kronecker product, so frmtmb has no block density for it.gr(g, cov = , dist = "student"),mm(dist = "student"): the t’s mixing variable is per level. With levels correlated throughA, or a row that loads several levels, the joint density over the field is not a multivariate t. brms samples the construction; the Laplace route has no density to expand.frm_compat()rows, each checked by running both packages:cens()xdisc,cens()xequidistant,vint()xsum_to_zeroandtrials()xcs_pred()are refused, in both packages.s()xprior,t2()xprior,mm()x|ID|,mm()xgr_cov,|ID|xmmc()andgr_covxmmc()work.mi()xse(),se()xskew_normal,cens()xmixture,thres()xmvbfandmi()x each family above work.mi()xasym_laplace,mi()xmixture,mi()xcens()andmi()xtrunc()are conditional on no density reading the value.importancexmi()andquadraturexmi()are conditional in the same way.trunc()xmixtureis refused.BREAKING.
ar(),ma(),arma(),cosy()andunstr()on a family with no residualy - muare no longer refused. They fit brms 2.23.0’s latent residual: one latent per row on the linear predictor ofmu, with a new parametersderrand prior class"sderr". The joint density at the conditional modes is an identity with brms’slog_probat three parameter points, within 4e-10. This holds on poisson, negbinomial, zero-inflated and hurdle poisson, bernoulli, binomial, Gamma, lognormal, beta and cumulative. The Laplace value is within 2e-7 of one computed from brms’s own program. frmtmb integrates the latents by Laplace on sparse coordinates: at 2000 rows in one series, the inner Hessian ofar(cov = TRUE)stores about 4000 entries, where brms’s coordinates would store 2 million.ar()withcov = FALSEon these families is brms’s moving average of independent latents, with each coefficient in (-1, 1). Its sum enters on the link scale where brms has a built-in density on that scale. Elsewhere it enters after the inverse link, as in brms’s program. A mean outside the family’s support has aNaNdensity, as Stan rejects it.frm()warns, with classfrmtmb_latres_laplace, where the Laplace approximation of a latent residual is biased: ordinal rows, or a first-order Laplace correction that moves logsderrorar[1]by more than 0.15, or cannot be read. Over 84 designs and 3130 fits it warned on 0.865 of the wrong fits and on 0.157 of the right ones. The rate was 0.14 to 0.17 on each set of designs. Above 3000 rows it reads the rows’ share of the latent, as the correction’s dense matrix is the cost that limit avoids. On bernoulli, 20 groups of 10,sderrreads 0.39 andar0.77 where the exact maximum likelihood is 1.10 and truth is 0.60. Onbeta_binomial()it reads 0.26 against 0.65, onzero_one_inflated_beta()0.28 against 0.59, onBeta()at phi 4 0.26 against 0.42. On negbinomial it is low by 8 to 15 percent, with coverage of logsderrdown to 0.66. The fit otherwise reports code 0. Poisson is fine down to a mean of 1, and lognormal does not warn: there the log density is quadratic in the predictor, and Laplace is exact. The condition has no units. The warning is about the estimates, socheck_sedoes not reach it, andsuppressWarnings(expr, classes = "frmtmb_latres_laplace")silences it.frm_sample()samples the same model and agrees with brms.?frmtmb-autocorhas the table.BREAKING for a call that fit in the previous development build.
frm()refusescov = FALSEwhere the AR part adds after the inverse link. This covers Gamma, Beta, Weibull, exponential, inverse Gaussian,beta_binomial(),zero_one_inflated_beta(), any non-canonical link, acens()ortrunc()response.frm_sample()fits it. A latent can carry the mean out of its range, and no number from the fit says whether Laplace reads the likelihood: on benign designs 54 of 120 fits read true. Beta at phi 50 read true at a margin of 6.6. Gamma with a log link at 23.6 readar-0.86 for 0.47. At low signal,beta_binomial()at 10 trials stops at a nonzero code on 20 of 20 with a gap of 9.0 nats. At low signal,zero_one_inflated_beta()readssderr0.28 for 0.38 with code 0. One Gamma fit reported a log likelihood of +3.39e18, and brms’s own chains struggle on these forms, with up to 3507 of 4000 divergent.On a link that does not hold the mean (identity, inverse,
1/mu^2), undercov = TRUE,frm()refuses after the fit. It refuses if the mean stands less than 2 marginal sds of the latent from the link’s edge. There 14 of 150 fits read true against the exact marginal, some inner starts readNaNand coverage of logsderris near 0. From 2 up there is noNaN, and 0.56 of 530 fits read true, in line with links that hold the mean.Gamma()at shape 50 readssderr0.309 against an exact 0.312, andpoisson("identity")at a mean of 50 reads 3.79 against 3.73.A dispersion the rows do not ask for no longer reaches an unbounded objective beside a latent residual. Before, a log likelihood read +6.5e17 on
beta_binomial(), +1.9e30 onBeta()and +4.1e7 onzero_one_inflated_beta(). A constant dispersion is held at a log of 15, and a dispersion with a predictor readsNaNabove a log of 25. A fit at the box warns with classfrmtmb_latres_dispersion, naming the term.frm()refusesmi()on the response beside a latent residual: the imputed values and the latents would share one inner problem, which no measurement has read.frm_sample()is not refused by this check.frm()refusesar(),ma(),arma()andcosy()undercov = TRUEif every group has one row, where brms builds the model. The correlation is not seen in the data, and only its product withsderrenters.frm()refusesasym_laplace()with a latent residual, where brms builds the model. The density has a kink in its mean, so the Laplace approximation has no maximum.frm_sample()samples the model.A fit at a limit of a latent residual says so: a message of class
frmtmb_boundary_fitthat names the limit,NAin place of a standard error of 0 insummary()andconfint_varcorr(). It also gives a row indiagnose(), and brms’s names in place oftheta_k. A limit is read from the fit’s numbers:ar[1]at 0.9992 on 2000 rows is well determined and reports its standard error. Atar[1] = 1withsderr = 0the message says the data read ascosy()or(1 | group); atar[1] = -1it says one size per group, alternating by row.predict()holdssderrand the term’s coefficients at their estimates when it draws the parameters of a latent residual. A draw oflog(sderr)with a standard error of 10 multiplied the latents byexp(30). This gave 3.7e304 on a poisson fit whose largest count is 10.brms’s refusals hold on these families, in brms’s words:
cov = TRUEabove order one, an MA part undercov = FALSE,fcor()and a nonlinearmu. A term on a multi-category family or a mixture says “Explicit covariance terms can only be specified on ‘mu’”.The methods read the latent residual as brms reads
err:fitted(),frm_linpred(),residuals(),predict()andsimulate()on the fitted rows, underre_formula = NAtoo. On new rows undercov = TRUE,predict()andsimulate()draw new latents; undercov = FALSE, new rows are refused in brms’s words.fitted()on new rows is the mean over the latent at each row’s marginal sd, as brms’sposterior_epred()is, andmarginaleffectsreads the same mean.frm_linpred()on new rows stays at a latent of 0.conditional_effects()andemmeans()drop the term, as brms’s do.Above 3000 rows
fitted(),residuals()andfrm_linpred(se.fit = TRUE)withnewdata = NULLon a latent residual giveNAfor the standard error, with a message of classfrmtmb_latres_se_skipped. This is because it would solve the joint precision as a dense matrix: 6 to 12 s at 2000 rows, 80 s at 4000, 984 s at 8000.options(frmtmb.latres_se_rows = )moves the limit.A decision, not a defect. Under
cov = FALSEfrmtmb’sfitted(),log_lik()and row densities follow the model. In contrast, brms 2.23.0’sposterior_epred()andlog_lik()leave out the AR sum of its own Stan program. Summed over the rows, brms’slog_lik()misses itslog_probby -6.78 nats on poisson and -113.48 on Gamma with a log link. This is atar0.6 andsderr0.5. The means differ by up to factors of 1.62 and 5.18. frmtmb’s row densities add up tolog_prob, so the two packages differ on this form.default_prior(),prior_summary(),variables(),summary()andconfint_varcorr()carry brms’s names and bounds on these terms:ar[1],ma[1],cosy,cortime__<t1>__<t2>andsderr.bf(..., nl = TRUE, autocor = ~ term)and+ acformula(~ term)fit a nonlinear model with a residual correlation term, as brms does. The log density equals brms 2.23.0’s Stan program forar(),ma()andarma()withcovTRUEandFALSE,cosy(),unstr()andfcor(), on both routes. A term written in the body is still refused, and the message now says to useautocor =.bf(y ~ 1, autocor = ~ term) + nlf(mu ~ ...)andbf(y ~ 1) + nlf(mu ~ ...) + acformula(~ term)take the term too.Under
cov = FALSE,fitted(),predict()andlogLik()of a body follow the model, on the one-step mean, as they do for a linearmu. This diverges from brms 2.23.0 on purpose: brms’s Stan program has the ARMA term. By contrast, itsposterior_epred()is the body alone. Itslog_lik()is the density at an ARMA coefficient of 0, 34.9 nats from its ownlog_prob()on one model. brms’s post-processing for a nonlinear formula does not read the term.nl = TRUEoncategorical(),multinomial(),dirichlet(),dirichlet2(),logistic_normal(),dirichlet_multinomial()and amixture()writes the one body into every location. It has one set of nonlinear parameters, as brms does. It was refused. The log density equals brms 2.23.0’s Stan program.fitted()works on these fits and on a body under an ordinal mixture; both were refused.A nonlinear body can call the functions of the Stan language that brms’s bodies call, which brms’s nonlinear vignette does:
inv_logit(),logit(),inv_cloglog(),Phi(),square(),cbrt(),exp2(),pow(),inv(),inv_sqrt(),inv_square(),hypot(),log1m(),step(),log1p_exp(),log1m_exp(),log_diff_exp(),log_sum_exp(),log_inv_logit()andlog1m_inv_logit(). They have Stan’s values and derivatives, on the tape and off it. A function of the same name found from the formula’s environment comes first, which includes the packages on the search path, except forstats::step().x^bandpow(x, b)in a body, with a dataxthat is 0 on a row, have Stan’s finite gradient.a * x^bstopped with “NA/NaN gradient evaluation” there, where brms fits it.A nonlinear fit under a link that cannot evaluate a body at 0 now starts.
frm(bf(y ~ b0 + b1 * x, b0 + b1 ~ 1, nl = TRUE), family = Gamma())and the same underinverse.gaussian()stopped with “failed from its default starting values”.frm()now moves one intercept the body reads to where the body equals the response mean, and says so in a message. A body undefined at its start, such aslog(a) + b1 * xunderGamma(), keeps the error that asks for a start. A start or a located prior stands.mixture(Gamma(), Gamma())moves the same way.pp_mixture()takes one response of a multivariate mixture, with brms’s refusal of none or several. Before, it refused every multivariate fit.insight::get_predicted(predict = "link")reads one predictor per row, the shared part of acs()term, because it returns one number per row.emmeans()of a model with ame()term no longer leaves the term out of the mean. Its design route read the intercept with nob * mean(x). It now takes the grid route, asmo()andmi()do.frm_curve()no longer refuses a grid with a row that has a standard error of exactly 0 on both routes of its covariance check. An example isx = 0underdpar = "mu"of an ordinal fit. It read the agreement as a disagreement of 1.New sampling-API exports for frmtmb.sample:
ce_by_response(),resp_refuse_single()andupdate_merge_prior().frm_compat():cs_pred()xfrm_curveis now “works”, where it was “untested”;me()xpredictandme()xemmeansare now “works”, where they were “conditional”.?"frmtmb-gp"is a new help topic. It states the nugget, the warning of classfrmtmb_gp_nugget, what to do about it, and the order of the field.frmtmb_control(check_mi = c("warning", "ignore", "stop"))sets whatfrm()does when the Laplace approximation of a readmi()value moves an estimate.options(frmtmb.latres_dense_rows = )sets the row limit above which a standard error refuses to read the whole joint covariance of a latent residual. The limit is 20000 rows. The size is16 n^2bytes: about 200 GB at 100000 rows, whereconditional_effects()used 3 GB at 12000 rows.
Bug fixes
frm(importance =)no longer stops with “did not converge … the corrected negative log-likelihood ROSE” on the models it exists for. It stopped on 162 of 648 fits over six families, three block shapes and three group sizes at 64 to 2000 draws. Of the others, 204 used every round. Now all 648 fit, and none uses every round. The fit used to freeze the proposal in place while the optimizer moved. A step in an intercept then moved the integrand under draws that stayed put, and the curvature the optimizer saw was a sample statistic: on a binomialy | trials(10) ~ x + (1 | g)of 10 groups of 20, 5 to 70 over seeds at 200 draws, and negative at one, against an exact 34. Where it came out too small, the rounds diverged, with effective sample sizes of 0.998. The proposal’s mode and variance now follow the optimizer to first order around the anchor, in steps measured in units of the predictor. So a fit does not read a column’s center or scale. The standard normal draws stay fixed, so the objective stays smooth, and at a given anchor each parameter value has an unbiased estimate. On that model at 200 draws, one round gives an intercept of 0.036581 and a log-likelihood of -418.1106 (Monte Carlo standard error 0.0049). For comparison, quadrature gives 0.036585 and -418.1121. Laplace gives -418.1233. Over the 648 fits the median round count falls from 4 to 1. The total time is about a third less where the old fit took 3 to 5 rounds. The 162 fits that used to stop now run to the end. The median error against the exact maximum falls from 0.0041 to 0.0002 in the fixed effects. It falls from 0.017 to 0.001 in the log standard deviations. Where both take 2 rounds, a correlated block costs more per evaluation: on 5000 groups, a gradient up to 1.8 times at 3 coefficients and 1.25 times at 2, a tape up to 2 times. A freeze at 40 fixed effects costs 3.6 to 5.6 times, and a whole fit there up to 15 percent more.confint(method = "profile")on animportance =fit gets bounds close to quadrature’s on designs where it used to warn that the proposal had stopped covering the integrand. On a scalar intercept over 40 groups of 4 binary rows at 500 draws, the bounds on the log standard deviation are -0.5115 and 0.7558. By comparison, quadrature’s are -0.5146 and 0.7576. There is no warning.frm(importance =)starts its rounds from where the Laplace route ends. It skipped the restart from a larger standard deviation, so a false collapse of a variance component stayed: every weight is equal there, so nothing left it. On an uncentered ordinal,w = 5 z + 2000, 15 of 40 fits stayed at a log standard deviation of -6.7 to -130. They read 0.15 to 3.33 nats below the Laplace fit.frm(importance =)fits a response in large units. Ony x 1e6a gaussian fit stopped on 6 of 8 data sets with “Report this model”. This is because the prior’s precision was read as a difference that rounded: 0.29 wrong at a standard deviation of 1e7, and 0 at 1e8. It now reads the precision where the density has moved by about one nat.frm(importance =)honors a prior. It used to stop on any prior with “could not reproduce this model’s joint log-density … Report this model”, thoughfrm_compat("importance", "prior")said it works. The prior reads the outer parameters only, so the objective adds it. Againstquadrature = TRUEwith the same prior, on a binomial(1 | g), the objective agrees within its Monte Carlo error. The estimates agree within 0.05 of their standard errors.A constant distributional parameter, as in
bf(y ~ x + (1 | g), nu = 5)withimportance =, evaluated the wrong model on the path that reads the weights. It readnufromsigma’s value, so the effective sample sizes and the Monte Carlo standard error described that other model. They read 0.50 and 0.102 where the right values are 0.98 and 0.015. The fit’s own objective was right. The defect dates fromimportance =itself, in 0.50.0.The numbers that
vignette("reinforcement-learning")andvignette("bayesian-cognitive-modeling")quote forimportance =are measured again. They came from the frozen proposal. On Exams_2 the Laplace error is 0.0002 nats, not 0.013, and direct numerical integration confirms it. On the 40-subject learning model at 100 trials the correction moveslog sd(alpha)by +0.12, not +0.36, and all six datasets complete.frm()refuses a nonlinear body the likelihood does not read: a family whose density does not read the dpar the body computes, or a parameter whose formula has terms and whose design came out empty. Both fitted without the body before. The refusal needs the body’s value to have moved while the objective did not, so a flat or saturated start does not reach it.cs()in a nonlinear body is refused with a sentence that says why. It was R’s “could not find function”. brms 2.23.0’s compile fails on it. A helper calledcs()that the formula’s environment finds is the user’s, and it is called.conditional_effects()on an ordinal body draws its Wald band through the body’s Jacobian, andband = "boot"works. Both stopped before.frm_linpred()andfitted()on a nonlinear body underna.excluderead the kept rows of a nonlinear parameter. Before, they mixed padded rows into the body’s data columns.gp(x, k = )has no nugget, but it is not a remedy by itself:kmust be large enough for the field. Onsin(8 * x)over 0 to 6 at noise 1e-3,k = 30stops at frequency 6.5, readssigma720 times high and reports convergence, with no warning.k = 60reads brms’ssigma.A gap of 94 to 1127 nats between
frm()and brms on an exactgp(), read earlier, was in the test harness: frmtmb sorts a term’s positions and brms keeps first appearance, and the translator did not reorder them.fitted()on a new row of a latent residual holds the latent’s draw inEst.Error. TheQcolumns are read from that draw, as on ame()row. On poissonEst.Errorread 0.133 where the draws’ standard deviation is 1.005. A row with both ame()value and a latent residual reads one mean and one variance over the two. Beside another group-level block it reads the joint covariance, up tofrmtmb.latres_dense_rowsrows: 14 seconds and 6.4 GB at 20000 rows. Above that its standard errors areNA, with a message of classfrmtmb_latres_se_skipped.(1 | p | gr(g, cor = FALSE))beside(1 | p | g)builds two uncorrelated blocks, as brms does, in either order.frm()refused it. A key writtenid =is still refused, since brms reads the first term’s flag there.frm_curve()at its default reads a row ofNAasNA, and the other rows answer. Before, it stopped with “infinite or missing values in ‘x’”.
Limits
Off the gaussian the start reads a working response, and it is not certain: on 150 fresh rows 13 stopped short by more than 1e-3 nats with no warning, against 51 on 0.78.0 (worst 420). Most are under 2 nats at low information, on bernoulli, Poisson and binomial, and some of those 0.78.0 reached. On a step-shaped field the gaps are larger: beside
ar()26 nats where 0.78.0 reached the maximum, withstudent()11 and 6 where 0.78.0 was 1.6 and 2.8 short. A nonlinear body is 98.5 short on both builds on a trend plus a wiggle. Astartforthetais how to check a fit that matters.sigmanear1e-6 * sdgp, where a nugget of1e-12reaches the noise, is the new limit, three orders below the old one. Theresigmacan read low. On 200 rows the fit’ssigmawas under half the true value on 92. A new warning of classfrmtmb_gp_nuggetnames this: it fired on all 92, on every row ofsigmaunder1e-6 * sdgp, and on none above3e-6 * sdgp. On 3 of 79 and 10 of 67 rows it fired withsigmawithin 0.8 to 1.25 of the truth. There the noise is at the nugget. Withse(), or where positions repeat,sdgpand the length scale can read low. On a non-gaussian family it did not fire on 18 rows, up to log mean 16 and 1e7 trials.Below about
1e-4 * sdgpnlminb also stops at false convergence, more often now since it starts at the maximum.fit$opt$convergenceanddiagnose()keep the code. The warning stays quiet only when a Newton step gains nothing and the stop reads the marginal’s own maximum tograd_tol. Only the warm start of a gaussian field can say so: on 144 rows it quieted 81 of 96, of which 3 were short, worst 0.81, at the marginal’s own second maximum. On 160 rows with a strong trend it quieted 37 of 66, of which 2 were short, worst 1.44. Every other stop still warns.On noise-free
sin(2 * x), 60 rows, the fit is 0.003 nats short of the marginal’s maximum, with its warnings. The fitted means at observed positions hold, and at new positions they are 2.6e-7 off. Standard errors at new positions read 1.1 to 2.4 times the plug-in standard error at the maximum.
Costs
A gaussian fit with an exact
gp()takes longer than the centered one did.frm()took 0.17, 2.1 to 2.2, 28 and 18 seconds. This is fromy ~ gp(x)on 60, 200 and 500 distinct positions, and on 1000 rows at 500 positions. Each ran one at a time on an unloaded machine. Before, it took 0.06, 0.64, 7.4 and 4.8: about 3, 3.4, 3.8 and 3.8 times. About half of that at 500 positions is the start, 13 of 28 seconds and 9 of 18, which saves the descent. The rest is the tape’s build, inner solve, Hessian and standard-error check. The inner Hessian isI + sd^2 L' W L, dense in the positions, and TMB’s tape reads it column by column, so the gradient grows faster than the value. Peak memory forfrm()andsummary()went down, from 0.89 to 0.73 GB at 500 positions. A saved fit grows, from 25 to 36 MB there.On a fit with an exact
gp(), the Hessian’s steps for coefficients are in y’s unit. A standard error’s row under its noise floor is flat only if it reads no curvature along its own parameter at two steps. An(Intercept)aty * 1000read flat and took every fitted standard error with it. Other fits keep their Hessian and their verdicts bit for bit.
frmtmb 0.78.0
The main changes:
- BREAKING: brms’s prior classes
meanme,sdmeandcormeof ame()term are honored, as in brms 2.23.0. A brms prior table passed tofrm()now applies itscormerow. - BREAKING: in a
default_prior(route = "sample")table, a group row prints brms’s(flat), so an edited class row reaches the groups. - New sampling-API export
predict_rescor_draw(), the joint predictive draw of aset_rescor(TRUE)model, and arespargument torescor_row_loglik(). - Update the extensions together with frmtmb. frmtmb.sample 0.25.0 needs frmtmb 0.78.0.
Breaking changes
brms’s prior classes
meanme,sdmeandcormeof ame()term are honored, with brms 2.23.0’s meaning. Before,set_prior()refused the three classes by name, and a brms prior table lost itscormerow, which is brms’slkj(1). Now that row applies. So a brmsget_prior()table passed tofrm()asprior =gives another MAP fit on a model with correlatedme()terms. Thefrm_compat()rowme()xprioris now “works”, where it was “conditional”.In a
default_prior(route = "sample")table, a group row reads(flat), as brms prints its""there: thesdandcorrows with agroup, the newcormerows with agroup, and a level ofthres(gr = ). Before, these rows printed the class row’s default. Passed back asprior =, that printed default applied as an explicit prior, so an edited class row did not reach the groups: a classcormerow edited tolkj(4)sampledlkj(1), with a Kolmogorov-Smirnov p-value of 1.3e-32 against thelkj(4)correlation. Now the edit reaches every group, as in brms. The class rows and the coefficient rows do not change.
New features
set_prior(class = "meanme")is a density on the latent mean of ame()term.class = "sdme"is a density on the latent standard deviation itself, with the log-Jacobian of the log map, as for class"sd".coefnames oneme()variable ("mex"forme(x, sdx)).class = "corme"takeslkj(eta)on the correlation matrix of the terms that share agr =, andgroupselects thatgr =variable. As in brms, the three classes take noresp,dparornlpar.meanmeandsdmetake nogroup, andcormetakes nocoefand no bound. A model withoutme()terms refuses them.default_prior(),validate_prior()andprior_summary()list brms’s rows for these classes.meanmeandsdmehave a class row and one row perme()variable, andsdmeis bounded below by 0.cormehas one row, plus one pergr =variable of a correlated set. On five model shapes the rows and bounds are those of brms 2.23.0’sdefault_prior().default_prior()andvalidate_prior()answer ame(x)written without its measurement SD, as brms’sdefault_prior()does.frm()refuses it, as brms refuses it when it reads the data. Before, both tables refused it.With a proper prior on every class, the posterior density of a
me()model equals brms 2.23.0’s Stanlog_probup to two terms. These are the Jacobian of the map between the two parameterizations and brms’slog(2)normalization of each half-Student-t. We compared three models at four points each: oneme()term, two terms withgr =and acormeprior, and a term beside(1 | h). The constant agrees to 6.4e-12 relative and the gradient to 4.3e-10 relative.predict_rescor_draw()is exported. It is the joint predictive draw of the responses of aset_rescor(TRUE)model, whichpredict()makes. frmtmb.sample’sposterior_predict()now draws through it, so the two cannot drift apart.rescor_row_loglik()takesresp, the responses whose joint density to give, as brms’slog_lik(resp =)asks. The correlation matrix follows the order ofresp.
frmtmb 0.77.0
The main changes:
- BREAKING: under the covariance form of
ar(),ma()andarma(),sigmais the innovation sd, as in brms 2.23.0. A fit saved by 0.76.0 or earlier with such a term is refused where an answer readssigma. - BREAKING: each random-effect parameter and the residual correlation of a multivariate model have one prior slot, as in brms. The named-list prior spelling is refused.
set_prior("")is brms’s(flat)on its slot. -
mo()takes a variable with one or two values, as brms does. - marginaleffects reads
vcov =and brms’s prediction arguments, and gives finite standard errors on a fit with an emptymo()category. -
build_objective()reads aprior_onlyfield, forsample_prior = "only"in frmtmb.sample. The sampling-API functionautocor_innov_sd()is removed. - Update the extensions together with frmtmb. frmtmb.sample 0.24.0 needs frmtmb 0.77.0.
Breaking changes
Under the covariance form of
ar(),ma()andarma(),sigmais the innovation sd, as in brms 2.23.0. Before, it was the marginal sd of a residual. The model was the same, but the parameter was not brms’s. On an AR(1) withar0.93,sigma(fit)was 1.078 and is now 0.406, against a brms posterior mean of 0.432. For an AR(1), the marginal sd issigma / sqrt(1 - ar^2). At any order, it issigmatimes the square root of the process variance at unit innovation variance. The logLik and the other estimates do not change.So
sigma(),VarCorr(),summary(),confint(),vcov(), the standard errors andfitted(dpar = "sigma")report the innovation sd. Undersigma ~ xthe coefficients ofsigmaare those of the log innovation sd, as brms’sIntercept_sigmaandb_sigmaare. Pearson residuals,simulate()and the pointwise log-density read the marginal sd, as before.A prior on class
sigmais a density on the innovation sd itself, and itslbandubbound it, as in brms. Before, the density was read at the marginal sd times a factor ofarandma. Albabove 0 or aubwas refused. Undersigma ~ x, a prior on the intercept ofsigmais now read at the log innovation sd, as brms reads it. Before, it was read at the log marginal sd.The standard error of
log(sigma)is the one of the innovation sd: 0.109 against 0.213 for the marginal sd on the AR(1) above. Its correlation with thearparameter falls from 0.88 to -0.34.ar(p = 2, cov = TRUE)and the other higher orders, which brms does not fit, readsigmaas the innovation sd too.conditional_effects(method = "posterior_predict")draws its band atsigma, so under these terms at the innovation sd. brms does the same: itsconditional_effects()drops every autocorrelation term. The band is narrower than before bysqrt(1 - ar^2)for an AR(1). On an AR(1) withar0.84, its half-width over 1.96 was 0.855 and is now 0.470.cooks.distance(),dfbeta(),dfbetas()andinfluence()read a coefficient vector that holds the log sd ofsigma, which is now the innovation sd. So their values change. On the same AR(1), the sum of Cook’s distances was 0.642 and is now 0.935.A fit saved by frmtmb 0.76.0 or earlier with one of these terms is refused where an answer reads
sigmaor its coefficients. These refuse:summary(),print(),sigma(),VarCorr(),confint(),vcov(full = TRUE),predict(),simulate(), pearson and devianceresiduals(), the influence measures, andhypothesis()onsigma.fitted(),frm_linpred(),emmeans()andconditional_effects()withdpar = "sigma"are refused, and so isconditional_effects(method = "posterior_predict"). Undersigma ~ x,fixef(),coef()andvcov()are refused too.insight::get_sigma(),get_parameters()andget_varcov()are refused as well.Such a fit still answers where only the mean is read, because its model is the same:
logLik(),AIC(),fitted(),frm_linpred(),emmeans()on the mean,residuals(type = "response"), and, with a constantsigma,fixef(),coef()andvcov().update()fits it again. Draws saved by frmtmb.sample 0.23.0 for such a model still report the old marginal sd assigmainsummary().For a current fit,
insight::get_sigma()gives insight’s default, as before:sigma()where it answers, andsqrt(deviance / df)undersigma ~ x, wheresigma()returnsNA. Only a saved fit with one of these terms is refused.The sampling-API function
autocor_innov_sd()is removed: no prior needs it now.mo()takes a variable with two values, or with one, as brms 2.23.0 does. Through 0.76.0 both were refused (“mo() needs at least 3 ordered categories”). Two values (brms’sJmo = 1) give a simplex of one weight, which is fixed at 1, so the term is its coefficient times the code. On 200 rows,y ~ mo(x) + zwithxin 1 and 2 has the log-likelihood oflm(y ~ I(x - 1) + z)to 12 digits.summary()reports the weightmox1[1]as 1 with an error of 0, as brms does. A two-level ordered factor is the same model. One value (brms’sJmo = 0) gives a term that is 0 on every row. Its coefficient is not determined by the data, so its standard error isNaN, with the usual warning. It is not counted in the degrees of freedom oflogLik(),AIC()andanova(), aslm()does not count a column it aliases.predict()holds it at its estimate. A group-level term on such a variable,(mo(x) | g), is refused: its column is 0 on every row. brms builds the model but cannot sample it: its Stan program rejectsJmo = 0, andfrm_sample()refuses it too. Soupdate(fit, newdata =)on new data where amo()variable has one value now refits by maximum likelihood, where brms’supdate()builds the model.marginaleffects’
vcov =argument reaches the fit. Through 0.76.0get_vcov()ignored it, so a user’s matrix,"HC3"or a cluster formula gave the model-based standard errors without a message:predictions(fit, vcov = 2 * get_vcov(fit))gave a ratio of 1 wherelm()gives 1.414214. Now a matrix overget_coef(), matched by its names, works. So do a matrix over the whole outer parameter vector, a function,FALSE,"HC0","HC1"and a one-sided cluster formula. marginaleffects maps"stata"to HC1. The whole-vector matrix is refused for a REML fit, whose vector has no fixed effects."HC0"and"HC1"equalsandwich::vcovHC()on a gaussian linear model, and~ gequalssandwich::vcovCL()on a Poisson glm. The bread is the observed information, so for a glm with a non-canonical link the sandwich differs fromsandwich’s, which reads the expected information: 1.128 times its standard error of the slope on 300 rows of aGamma(link = "log")model. The other strings are refused, each with its reason:"HC2"to"HC5"and"robust"need leverages, which frmtmb does not compute,"HAC"and"NeweyWest"a time order,"satterthwaite"and"kenward-roger"denominator degrees of freedom. Under a residual correlation term, such asar(cov = TRUE),"HC0","HC1","stata"and a cluster formula are refused too. Its density is joint over each group.vcov = TRUEworks there. See?frmtmb-marginaleffects.Take a fit whose
mo()term has a category that no row is in. Thereget_coef()carries that simplex as its cumulative weights at the observed categories (zeta1_cum1, …), not as its chart coordinates. So marginaleffects givesNaNonly to a prediction at an empty category, with one warning, asfitted()does. In 0.76.0 every standard error of such a fit wasNaN. On 200 rows with values 1, 2 and 5,predictions()at 1, 2 and 5 now gives 0.1222210, 0.1135415 and 0.1212624. These are the standard errors oflm()onfactor(x)with the maximum-likelihood sigma. Withvcov = "HC0"it gives 0.1224166, 0.1103619 and 0.1248376,lm()’s HC0.marginaleffects’ prediction arguments that frmtmb does not take are refused by name: lme4’s
re.formandallow.new.levels, brms’ssample_new_levels,ndraws,draw_idsandincl_autocor, and glmmTMB’szitype. marginaleffects warns that every argument of afrmtmb_fitis “not known to be supported”, so a dropped argument looked the same as one that works:re.form = NAreturned the conditional predictions.Each random-effect parameter, and the residual correlation of a multivariate model, has one prior slot, as in brms 2.23.0, which has no class
theta. A standard deviation is classsd(agp()term’ssdgp, andlscalefor its length scales). A correlation that an LKJ density fits is classcor, and a residual correlation is classrescor.set_prior(class = "theta", coef = "theta_4")on such a parameter is refused with the class to write, anddefault_prior()lists nothetarow for it. Until now classthetawas a second slot on the same parameter, and the one written later replaced the other with no message. On(1 + x + z | g),lkj(2)on classcorand thennormal(0, 1)ontheta_4kept the normal on one correlation parameter and no density on the other two. A MAP fit took those two to 559 and 1741. A classsdprior withcoef = "x"beside atheta_2prior was dropped. Classthetakeeps the parameters that no brms class names. Examples are atoep()correlation and a parameter of a structure that an extension registers (acar()dependence). With nocoefit covers only those. One correlation takes no bound any more. brms has none either: its program forlbon classcordoes not parse.A centered intercept has one prior slot, class
Intercept.set_prior(class = "b", coef = "Intercept")there is refused, as brms refuses it. It put a second density on the intercept, without the centering. ClassInterceptwithnlparis stored as brms’s slot of the same parameter, classbwithcoef = "Intercept". So the two spellings together are refused as a duplicate. Before, the one written later applied.default_prior(route = "sample")prints brms’s(flat)withlbandubfor a first-ordercov = TRUEarorma, forcosyand forlagsar, where it printeduniform(-1, 1). The table passed back asprior =gives the same sampled density as the defaults: at 300 points on each of six models the log density agrees to 2.3e-16 relative and the gradient exactly. Onfrm()’s route the same table is read as written: a(flat)row withlbandubis a box with no density, as every bounds-only specification is there. A box at the ends of the interval bounds nothing. Sofrm(prior = default_prior(..., route = "sample"))on such a model moves, anduniform(-1, 1)onarand(flat)withlb = -1, ub = 1give differentfrm()fits. Ony ~ 1 + ar(t, g, cov = TRUE)(6 series of 8) the internal ar estimate under the table is 0.8053, where it was 0.7249; maximum likelihood gives 0.8089. The prior object thatprior_summary()returns for afrm_sample()run carries the densities. Passed tofrm(), it still gives 0.7249, the mode of the sampled density.The named list of prior objects,
prior = list(theta_1 = prior_normal(0, 1)), put densities on internal parameters by name.frm(),update(),validate_prior()andfrm_simulate()refuse it, and say how to write the same density withset_prior(). brms has no such spelling. It gave a parameter a second prior slot beside its class: on(1 + x + z | g)infrm_sample(),list(theta_4 = prior_normal(0, 1))removed the defaultlkj(1)from all three correlations and left two of them with no density. A plain data frame passed asprior =is refused too, as brms refuses one, with a pointer toas.brmsprior(), which reads it. Theprior_normal()family of constructors stays: they are densitiesset_prior()takes. With the list goes the only prior on a single threshold’s internal parameter and on the free directions of sum-to-zero thresholds; brms has neither.set_prior("", class = ...)with no bound is brms’s(flat)on that slot, where it was refused.frm()applies nothing there, so a prior made only of such slots leaves a maximum-likelihood fit. Adefault_prior()or brms table row with an empty prior and no bound is read the same way, where it was dropped. Adefault_prior(route = "fit")table keeps that route through a row or column selection andsubset(). A priorlist read from it keeps it through+,c()andas.brmsprior(). So frmtmb.sample can refuse it.as.data.frame()drops the route. Soas.brmsprior()marks a data frame that holds frmtmb’s printed(flat).frm()drops either mark from the prior it keeps.c()of a prior specification with something that is not one, such as a density object or a named list of them, is refused. The message gives theset_prior()spelling to write. Before, it built a prior thatfrm()then stopped on with an internal error.c()now reads adefault_prior()table or a brms prior asas.brmsprior()reads it, with its route. That holds when a specification or a frmtmb table comes first. brms’sc()combines the rows the same way. With a brms table first, brms’s ownc()refuses the frmtmb specification.
New features
A prior density can be brms’s
(flat). The prior resolver reads it as no density on the parameter a class names, with the Jacobian of the map to the internal parameter. An LKJ density can be flat on the Cholesky factor. frmtmb.sample’sprior = "flat"uses both, so frmtmb.sample 0.24.0 needs frmtmb 0.77.0.build_objective()reads a third field on a copy of the frame,prior_only. Withprior_only = TRUEthe objective leaves out every response density. It keeps the densities of the group-level coefficients, of theme()latent values and their measurements, and of ami(sdy)response’s measurements. These are the terms brms 2.23.0 keeps outsideif (!prior_only)underbrm(sample_prior = "only"). A fit’s own frame never carries it. frmtmb.sample’sfrm_sample(sample_prior = "only")reads it, to sample the prior of the whole model.?frmtmb-sampling-apidocuments it.plot()of a hypothesis draws the density of its prior draws in the second ofcolors, as brms does, unlessignore_prior = TRUE. Draws made byfrmtmb.sample::frm_sample(sample_prior = "yes")carry them; a maximum-likelihood fit has none, and its plot does not change.
Bug fixes
predictions(),comparisons()andslopes()of marginaleffects pass brms’sre_formula,allow_new_levels,dpar(also brms’snlpar) andrespto the prediction. Through 0.76.0 they were dropped, sopredictions(fit, re_formula = NA)returned the predictions conditional on the group-level effects. marginaleffects 1.0.0 still warns that these arguments are “not known to be supported” for this class. It keeps a fixed list of known arguments per class. The arguments have their effect.predictions()on new data with a factor that has levels no row uses returned one row more for each level: 11 rows for 3 rows of a factor with 8 levels. marginaleffects adds a row for each level and removes those rows by their row id, which frmtmb’sget_predict()numbered again.vcov_cluster()gaveNaNin every entry on a fit that lost a parameter (amo()category no row is in, a variance at its boundary). The bread heldNaNin the lost rows. The bread is now the covariance conditional on the lost directions, and only the lost rows areNaN.prior =given a list ofset_prior()specifications, such aslist(sp, sp2), is refused with a pointer tosp + sp2andc(sp, sp2), as brms combines them. It gave a generic message about a list of length 2.
frmtmb 0.76.0
The main changes:
- BREAKING:
mo()codes an integer variable from its minimum, as brms 2.23.0 does. Every fit whose integermo()variable does not start at 0 changes. A fit saved by 0.75.0 or earlier with such a variable is refused. - BREAKING: each residual correlation parameter has one prior slot, and the prior tables bound the residual correlation classes as brms does. Under a first-order
cov = TRUEar(),ma()orarma(), a prior on classsigmais read at brms’s innovation sd. - The standard error of a prediction includes the uncertainty of the
mo()simplex. - New in the sampling API:
autocor_flat_priors(),autocor_bound_chart(),autocor_innov_sd()andautocor_row_loglik(). New seam slot:frmtmb_register_autocor(jacobian =). - Update the extensions together with frmtmb. frmtmb.sample 0.23.0 and frmtmb.spatial 0.3.0 need frmtmb 0.76.0.
Breaking changes
mo()codes an integer variable from its minimum, as brms 2.23.0 codes it: values 1 to 4 have a simplex of 3 weights and the codes 0 to 3. Sincemo()was added in 0.15.0, frmtmb coded the integers themselves. So the simplex had one weight for each value between 0 and the minimum that no row has. Every fit whose integermo()variable does not start at 0 changes, which includes every Likert item on a 1..k scale. On such data (300 rows, values 1 to 4) the maximum-likelihood coefficients were not identified: the fit had 6 degrees of freedom where brms’s model has 5, and 3 of 4 predictions had a NaN standard error.frm_sample()sampled that other model, with only ESS warnings: Intercept 0.912 (sd 1.330) against brms’s 2.098 (0.117). Now the fit on 1..4 is the fit on 0..3, with the same log-likelihood, coefficients and finite standard errors.frm_sample()agrees with brms within 2 Monte Carlo standard errors on every parameter. An integermo()variable with only two values, such as 1 and 2, is now refused (“mo() needs at least 3 ordered categories”). 0.75.0 fitted it with an empty category 0, so its coefficients had no standard errors. brms fits it as a linear term in the code.mo()takes negative integers, as brms does; 0.75.0 refused them.New data for an integer
mo()variable are coded from the fitted data’s minimum. A value that is not a whole number is refused, as brms’svalidate_newdata()refuses it. So is a value below the fitted minimum or above the fitted maximum. Through 0.75.0 a value between 0 and the fitted minimum was predicted without a message, and a fractional value was rounded to the nearest category: on a 1..4 fit, 0.6 was predicted as 1 and 4.4 as 4. Somarginaleffects’avg_slopes(variables = "k")was 0, because its finite difference never left a category; it is now refused with this reason.avg_comparisons()between two categories works. A missing value, a factor or a character vector in new data is refused with its own reason.mo(mi(x))is refused at the fit, as brms refuses it.conditional_effects()holds a numericmo()variable atround(median())and moderates withround(median +/- mad), as brms does. It held the variable at the mean and the mean +/- sd, which the rounding above then moved to another category. So a display was off by a whole category, and its labels named values the model was not evaluated at. The grid of amo()effect is the fitted term’s categories, which in asubset()response can be fewer than the data’s.emmeansrefuses a reference grid that holds an integermo()variable at its mean, as brms does, and names the remedy:at =orcov.reduce = list(k = function(v) round(median(v)))for the variablek, which leaves every other covariate at its mean. It reported the mean at the rounded category under the label of the mean.The standard error of a prediction includes the uncertainty of the
mo()simplex, infitted(),predict(),frm_linpred(),conditional_effects(), emmeans and marginaleffects. Since 0.15.0 the delta method left it out. On 200 rows with the values 1, 2 and 5, the standard errors at 1, 2 and 5 were 0.121, 0.0987 and 0.121. They are now 0.122, 0.114 and 0.121, and brms’s posterior sds are 0.124, 0.116 and 0.123. A prediction at a category that no row has, inside the range (3 and 4 here), now has aNaNstandard error. The fit’s warning says so. Through 0.75.0 it was finite (0.0859 and 0.0950). marginaleffects reads the simplex throughget_coef()andget_vcov(). On 300 rows ofy ~ mo(k) + z,predictions()at k = 2 and 3 gave 0.0846 and 0.1035. The exact standard errors are 0.1220 and 0.1142, and it gives those now. On a fit with an empty category marginaleffects cannot leave the undetermined direction out, so its standard errors areNaNthere, with a warning.dfbeta()and insight keep the coefficients alone.A fit saved by frmtmb 0.75.0 or earlier is refused when its integer
mo()variable does not start at 0. The message says to fit the model again. Every method that reports an estimate, a degree of freedom, a standard error or a comparison refuses it:fixef(),coef(),logLik(),AIC(),BIC(),vcov(),confint(),summary(),anova()anddrop1(). So dofitted(),predict(),simulate(),residuals(),conditional_effects()andfrm_sample().update()fits it again. A saved fit on data that start at 0, or on an ordered factor, is the same model and works as before.One residual-correlation parameter has one prior slot, as in brms.
default_prior()listed each parameter of anar(),ma(),arma(),cosy()orunstr()term twice: under its class (ar,ma,cosy,cortime) and asclass = "theta", coef = "thetaac_1".validate_prior()andfrm()took a prior on each, and the fit applied the one written last. brms 2.23.0 has the class row only. Now the internal row is not listed, and a prior on it is refused with the class to use: “class =”theta”, coef = “thetaac_1” is the internal parameter of ar, which has a class of its own, as in brms. Write the prior with class = “ar”“. A multivariate model’s refusal names theresptoo. This is the rule thatsar()’slagsargot in 0.75.0. It is now one rule for core’s classes and for each bounded parameter of a term registered withfrmtmb_register_autocor().class = "theta"still reaches a parameter that no class reaches: a random-effecttheta_1, athetar_1, and a parameter of a registered term that declares no interval. So two kinds of parameter take no box now. The internal parameters of acov = TRUEar()orma()of order two or more take none; brms’scov = TRUEform is of order one only. A single correlation of anunstr()term takes none either:class = "theta", coef = "thetaac_3"was the one route to it, andclass = "cortime"takeslkj()on the whole matrix and nolb/ub, as in brms.The prior table bounds the residual-correlation classes as brms does.
default_prior()listslb = -1andub = 1on a first-ordercov = TRUEarorma, as brms 2.23.0 does, and no bound undercov = FALSE. Forcosyit lists the windowfrm()fits it on,-1/(d - 1)to 1 fordtime points. Onroute = "sample"it lists brms’s 0 to 1, where frmtmb.sample now samples it.A prior on
sigmaunder a first-ordercov = TRUEar(),ma()orarma()is read on brms’s sigma, the innovation sd. frmtmb’ssigmathere is the MARGINAL sd of a residual, andsummary(),sigma()andVarCorr()report it. brms’ssigmais the innovation sd, smaller byc = sqrt(1 - ar^2),1 / sqrt(1 + ma^2)orsqrt((1 - ar^2) / (1 + ma^2 + 2 ar ma)). On an AR(1) of 0.9 that is a posterior mean of 1.16 here against 0.43 in brms, for the same model. Before,set_prior("student_t(3, 0, 2.5)", class = "sigma")was a density on the marginal sd. Now it is brms’s density atsigma * c, with the Jacobian ofc, infrm()’s MAP penalty and in frmtmb.sample alike. Albabove 0 or aubonsigmathere is refused. A box on the innovation sd moves withar, so it is no box on the marginal sd frmtmb holds. brms 2.23.0 takes such a bound on its innovation sd (real<lower=0.2> sigma).
New features
Three functions join the interface for a sampling extension (
?frmtmb-sampling-api).autocor_flat_priors()gives brms’s flat default on each residual-correlation class that frmtmb holds through a transform.autocor_bound_chart()gives a chart that samples a bounded residual-correlation parameter through one logistic onto its interval.autocor_innov_sd()names the responses whosesigmabrms reads as an innovation sd. frmtmb.sample uses all three.New in the sampling API:
autocor_row_loglik(fit, dpv, resp), the pointwise log-density of a response with a residual covariance term. These terms are the covariance form ofar(),ma()andarma(),cosy(),unstr(),fcor()and a registered term. Each row is brms 2.23.0’s leave-one-out conditional density given the other rows of its group, read off the precision matrix. On 18 cases (every term, gaussian and student, ragged groups, gaps,sigma ~ x) it equals brms’slog_lik()at the same parameters to 2.0e-15 relative. One draw costs one small inverse per pattern of time points. On 400 groups of 10 rows a draw takes 0.64 ms, against 17.8 ms for brms’s per-group dense inverses. Forfcor()on 600 rows it takes 2.6 ms, against 97.5 ms. The values do not sum to the log-likelihood.frmtmb_register_autocor()takesjacobian =, the Jacobian of the term’s whitening map, from which the pointwise log-density reads the precision matrix. A sparse one costs its nonzeros per draw. Without it, frmtmb invertsunwhiten()of the identity, which is exact and dense: 0.5 s and 48 Mb per draw at 1000 rows, 1.7 s and 155 Mb at 2000. Above 1000 rowslog_lik()warns once per call, with classfrmtmb_autocor_dense_fallback. A givenjacobianis checked against the map when the frame is built. The check runs at the start and at a generic point: sigma not 1 and varying by row, and every parameter moved off the start. A wrong one is refused by the term’s name.
Bug fixes
A bound at an end of a residual-correlation parameter’s interval is accepted, and bounds nothing, since the parameter never reaches that end. Before,
frm()refused it:set_prior("", class = "ar", lb = -1)stopped with “lb = -1 is outside (-1, 1)”. So thedefault_prior()table of asar()model, whoselagsarrow lists the ends of its interval, did not pass back asprior =in 0.75.0. Past the end, a bound is still refused.?frmtmb-autocorsaid thatset_prior()cannot target the residual-correlation parameters; it has done so by their classes since 0.49.
frmtmb 0.75.0
The main changes:
- BREAKING: coefficient names and orders follow brms 2.23.0. A term that calls
exp(), or another function named like a covariance structure, has brms’s name (expx, not.frm_expx).default_prior()lists each coefficient by brms’s name (IxE2,polyx21), and the prior tables sort their rows as brms sorts them.fixef()orders nonlinear and multivariate rows as brms does. - BREAKING: brms’s
bhaz()addition term is in frmtmb.survival. Without that package,bhaz()stops with a message that names it. - BREAKING:
set_prior()accepts adirichlet()row for every class.validate_prior()andfrm()refuse it when the class is not a simplex of the model. - New: brms’s
fcor()residual term is in frmtmb, and brms’ssar()term is in frmtmb.spatial. - New seams for extension packages:
frmtmb_register_aterm(option =)for an addition term that sets an argument of the family, andfrmtmb_register_autocor()for a residual correlation term. With them comefrmtmb_autocor_pars(),frmtmb_covstruct_info()andfrmtmb_register_covstruct(full_rank =). - A saved fit that needs extension packages this session cannot load names every one of them in one refusal.
- Update the extensions together with frmtmb. frmtmb.survival 0.2.0, frmtmb.spatial 0.2.0, frmtmb.sample 0.22.0 and frmtmb.latent 0.7.1 need frmtmb 0.75.0.
Breaking changes
A
dirichlet()prior on a parameter that is not a simplex is refused at one point, however it is written: where the prior meets the model, invalidate_prior()andfrm(), before any fitting. Through 0.74.0,set_prior("dirichlet(2)", class = "Intercept", dpar = "sigma")was refused byset_prior(), butclass = "sigma"was refused at the fit. Only the model can tell a declared simplex (frmtmb_family(simplex_pars =)) from a distributional parameter. Soset_prior()now accepts the row for every class, and the refusal repeats the spelling as written. brms 2.23.0 accepts both spellings inset_prior(),validate_prior()andstancode(), and stops at the Stan compile. The link-scale spelling of a declared simplex,class = "Intercept", dpar = "tw", is refused by name; writeclass = "tw". A brms prior object with such a row is refused the same way. Before, it was refused as “a brms prior table has 1 row with no faithful frmtmb spelling”. Onclass = "theta", the refusal adds that brms’sthetaof amixture()is the Dirichlet on its mixing proportions, which frmtmb’sthetais not.brms’s
bhaz()addition term is frmtmb.survival’s. Without that package,time | bhaz(df = 6) ~ xstops with “The bhaz() addition term is in the extension package frmtmb.survival, which is not loaded”, asdec()names frmtmb.eam. Before, frmtmb refusedbhaz()as a term it does not support and pointed tocox(df =). With frmtmb.survival loaded,bhaz()sets the basis of thecox()baseline, as in brms 2.23.0.A term that calls
exp()has brms’s coefficient names. To keepexp(x)from being dropped as a covariance structure, the parser renames the call to.frm_exp(x). Since 0.15.0 that alias was the name of the design column. So the coefficient was.frm_expxinfixef(),vcov(),summary(),coef(),ranef()andVarCorr(). It wasb_.frm_expxinvariables()and in the draws offrmtmb.sample::frm_sample(), and.frm_exp(x)inconfint()anddefault_prior(). brms 2.23.0 names itexpxandb_expx, andset_prior(coef = "expx")andhypothesis(fit, "expx > 0")were refused. The same applies toI(exp(x)),exp(x):z, a group-level(1 + exp(x) | g), and a call of any other function whose name is that of a covariance structure. Now the alias stays in the parser: the coefficient isexpxeverywhere brms has it, and theconfint()row isexp(x). Of 40 formula terms compared with brms 2.23.0, 18 had the alias in their names.default_prior()lists each coefficient by brms’s name, andset_prior(coef =)takes that name. Thecoefcolumn held the design column, such aslog(xp),I(x^2)orpoly(x, 2)1, where brms 2.23.0 listslogxp,IxE2andpolyx21. Andset_prior()refused brms’s name where the rename is more than dropped parentheses, such asIxE2,polyx21orexpxD2. The design column name still works inset_prior(coef =), for class"b", class"sd"and acs()term alike. The two spellings name one slot, so writing both is refused as a duplicate. Before, class"sd"andcs()refusedcoef = "I(x^2)"and"poly(x, 2)2".The rows of
default_prior(),validate_prior()andprior_summary()are in brms’s order.default_prior()orders them as brms’s does, by response, distributional parameter, nonlinear parameter, class, group and coef, sobcomes beforecor,Interceptandsd.validate_prior()andprior_summary()order them as brms’s do, by class, group, response, distributional parameter, nonlinear parameter and coef. frmtmb’s own class"theta", which brms does not have, stays last. Before,Interceptcame first andcoraftersd, and only thesdrows ofvalidate_prior()were in brms’s order.A nonlinear parameter of a multivariate model has brms’s prefix. brms writes the distributional parameter, then the response, then the nonlinear parameter, so
aof the responseynisyn_a:b_yn_a_Interceptandsd_g__yn_a_Interceptinvariables()and the draws,yn_a_Interceptinfixef(). frmtmb wroteb_a_yn_Interceptanda_yn_Intercept, andhypothesis()refused brms’s names.fixef(),vcov()andsummary()of a fit list the rows in brms’s order in two more cases. A nonlinear parameter’s intercept keeps its place among its coefficients, because brms does not center the formula of a nonlinear parameter: forbf(y ~ a + b * x, a ~ exp(z), b ~ 1 + z, sigma ~ z, nl = TRUE)brms 2.23.0 givessigma_Intercept,a_Intercept,a_expz,b_Intercept,b_z,sigma_z, where frmtmb listeda_Interceptandb_Interceptfirst. And a multivariate model lists its rows response by response, as brms declares them:sigma_yn_Interceptbeforey_Interceptwhenynis the first response.
New features
frmtmb_register_aterm(option =)registers an addition term that carries no data and sets an argument of the family instead. brms’sbhaz(df = 6), for example, sets the baseline basis ofcox(). frmtmb calls the function with the family, the term’s call as written and the formula’s environment, and fits the family that it returns. A family whosefrmtmb_family(accepts_aterms =)does not name the term is refused before the function runs, as brms refusesbhaz()for a family other thancox(). frmtmb.survival uses it forbhaz().frm_compat()’s generic rows for addition terms now say what each kind of term does. A registered data term carries a column to the family’s density. An option term adds nothing to the likelihood and sets an argument of the family of its own response.fcor(M), brms’s fixed residual correlation term, is supported. As in brms 2.23.0, the residuals of agaussian()orstudent()response are one draw with covariancesigma^2 M. Under a predictedsigma ~ ...the covariance isD M DwithD = diag(sigma).Mcomes fromdata2, covers every row of the data, and loses the rows the model drops, as in brms; a vector is its diagonal. The log-likelihood equals brms’s Stan program to 7e-15 at frmtmb’s estimates (gaussian, with a brms gradient there below 3e-6). It equals it to 1.4e-14 at set parameter values withnu = 5(student). A brms fit of 64 rows agrees with the maximum likelihood fit to within 0.33 standard errors in every parameter.fitted(),residuals()(pearson divides bysigma * sqrt(diag(M))),simulate(),predict()andautocor_matrix()work on the fitted rows. They also takenewdatawith one row per row ofM, read in their order, as brms reads it. So a counterfactual (x + 1everywhere) works. When the fit dropped rows for missing values,newdatacovers them too, and a row with a missing predictor givesNA, as in brms.fitted()there equals brms’sposterior_epred()at the same parameter values. Othernewdatais refused, becauseMcovers the fitted rows only; brms refuses it too unlessnewdata2holds a matrix over the new rows.frm_linpred()takes anynewdata. frmtmb refusedfcor()by name before.brms 2.23.0 checks that
Mis positive definite withmin(eigen(M)$values <= 0), which passes a matrix with a negative eigenvalue. Its fit of such anMthen fails in Stan, where the Cholesky factor ofMis taken. frmtmb refuses everyMthat is not positive definite, before the fit.brms’s
sar()is now supported in frmtmb.spatial. Without that package,sar()in the formula or inbf(autocor = ~ sar(M))stops with a message that names it, where frmtmb refused it as unsupported before.set_prior()takesclass = "lagsar"andclass = "errorsar", as brms does.New seam
frmtmb_register_autocor(). A package registers a term that correlates the residuals of a gaussian or student response. The term is one affine map from the response to independent innovations, with its inverse. frmtmb writes the density, the mean, the draws, the residual covariance and the guards from the map.fcor()is frmtmb’s own term of this shape, and frmtmb.spatial registerssar()through it. A covariance structure (frmtmb_register_covstruct()) cannot express such a term: it adds a latent vector to the linear predictor, and a lag SAR changes the mean of the response itself.frmtmb_local_registries()saves and restores the new registry.The map gets one
sigmawhen the term setssigma_varies = FALSE.unwhiten()takes a matrix of innovations, one column each, so the pearson residuals andautocor_matrix()solve the term’s system once for every column. On a 50 x 50 lattice undersar()(2500 rows), they take 0.48 s and 4.3 s. One solve per column took 16 s each. When the model is built, frmtmb checks the map once.unwhiten()must give one value per row and one column per column of innovations.whiten()must give one innovation per row and one finite log-Jacobian.whiten(unwhiten(u))must give backuto a relative 1e-6. A map that fails is refused with the term’s name.A term’s
boundsslot states the interval of a parameter thatnaturalmaps bylower + (upper - lower) plogis(theta).set_prior()then takes the parameter’s name as its class, andlbandubnarrow the interval. frmtmb.sample’s default prior on it is flat on the interval, as brms’s onlagsaris. A class prior on a parameter withoutboundsis refused, with the internal parameter to write it on instead. A parameter withboundshas one prior slot, its class: its internal row is not indefault_prior(), and a prior on it is refused. New exportfrmtmb_autocor_pars()lists the parameters of a fit’s registered terms, with their intervals.A slot of
frmtmb_register_autocor()orfrmtmb_register_covstruct()can be a function of....New export
frmtmb_covstruct_info(): what the registry knows about one covariance structure (its package, whether it is a field, whether its levels are coupled, whether it is full rank). frmtmb.sample reads it instead of keeping its own list of field structures. A name that core lists as an extension’s ("car","spde") loads that extension. A residual correlation term’s name, such as"sar", is refused with a pointer tofrmtmb_autocor_pars().frmtmb_register_covstruct(full_rank =)states whether an effect a row owns can reach any value with a finite density: a covariance that is positive definite at some parameter value and a density that is positive everywhere. A structure that states nothing leaves the weights check open for it, because its covariance does not tell: a latent with bounded support keeps a weighted model’s maximum. A package whose structure is full rank should statefull_rank = TRUE.autoscale leaves the random slopes of every structure but
us()anddiag()as they are. That holds for frmtmb’s ownhomdiag()orcs()as much as for a structure another package registers. Nothing tells frmtmb that their parameters scale with the column. It now says so in a message that names the structure and the column. It said nothing before.
Bug fixes
A saved fit that needs extension packages this session cannot load names every one of them in one refusal (class
frmtmb_extension_missing), with one instruction: “This model needs 2 extension packages …: frmtmb.survival (the cox family of this model) and frmtmb.spatial (the car() structure of this model). Install them and run library(frmtmb.survival); library(frmtmb.spatial)”. Through 0.74.0 acox()fit with acar()term named one package at a time:summary()named frmtmb.survival andprint()named frmtmb.spatial, so the second was found only after the first was installed. Every method of a fit that reads the model’s families, structures or residual correlation terms checks them all first. Sohypothesis(),residuals()anddrop1()name every package too. A method that reads only stored values, such aslogLik(), still answers. The refusal for acox()fit an older frmtmb built names the other missing packages too. Where its new home, frmtmb.survival, is installed but cannot be loaded, the refusal says to reinstall it.fixef()of a fit with a part from a package this session cannot load refuses, by the package’s name. Through 0.74.0 it answered withEst.ErrorNA: it read the refusal as “no covariance”. The same was true of a savedcox()fit from frmtmb 0.73.0 with frmtmb.survival installed. Every handler that reads an error as “no answer” now lets this refusal through. Sopredict()no longer warns first that “the covariance of the estimates could not be recovered from the Hessian”. Andcooks.distance()no longer says that “all deletion refits failed”.predict(newdata = )names the reason for the model at hand. Through 0.74.0 it said “its draws are structured (a hidden state sequence, a group-level latent class, or a correlated residual)” for every model. Amixture_mvn()fit is none of the three. It now names the family that draws its whole response at once, or the residual correlation term, such asar(time = t, gr = g, cov = TRUE).print()of a curve (frm_curve(),frm_curve_deriv()) or a curve feature (frm_curve_feature()) takesdigits =, which reaches the printed rows as inprint.data.frame(). Through 0.74.0 it stopped with “print() has no argumentdigits”.drop1()andstep()see a term that callsexp(). With the alias,drop1()found no columns for the term: ony ~ exp(x) + z(200 rows) the.frm_exp(x)row hadDf 0and the full model’s AIC, 607.84, so the term looked free to drop. The row is nowexp(x), withDf 1and AIC 625.69, the AIC ofy ~ zrefitted. Andstep()dropsz, which lowers the AIC to 605.84; before, it kept both terms.re_formulacan name a group-levelexp()term.fitted(fit, re_formula = ~ (1 + exp(x) | g))ony ~ exp(x) + (1 + exp(x) | g)stopped with “matches no group-level term of this fit”, because the fit’s term was(1 + .frm_exp(x) | g). It now gives the full prediction, which is also the prediction of the same model on a column holdingexp(x).A smooth’s unpenalized column written in both spellings is one prior slot.
set_prior(coef = "s(x).fx1")andset_prior(coef = "sx_1")name the same coefficient. But in 0.74.0 the pair was taken as two specifications, and the later one replaced the earlier without a word: withnormal(3, 0.001)on the first andnormal(-3, 0.001)on the second,validate_prior()listed only the second and the fit put the coefficient at -3. The pair is now refused as a duplicate, as two specifications for one slot always are.A multivariate model whose second response’s nonlinear formula calls
exp()fits. In 0.74.0 such a model, for examplemvbf(f1, f2)withf1 = bf(yn ~ a * exp(b * x), a ~ 1, b ~ 1, nl = TRUE)andf2 = bf(y ~ c + d * x, c ~ exp(z), d ~ 1, nl = TRUE), stopped with ‘could not find function “.frm_exp”’.The weights check refuses a weighted likelihood with no maximum. It named its full-rank structures in a list, so it failed open for every structure another package registers. It now reads each structure’s
full_rank. Take a Gamma model with weights 1.5 and a registered iid block on one level per row that statesfull_rank = TRUE. It is refused now, as the same model with(1 | obs)is. Before, it fitted to a logLik of 849 with a shape of 5e14 and two warnings. A registered structure that is not full rank, or states nothing, is not refused.The Student-t latents of
gr(dist = "student")were not full rank for the weights check, so it failed open for them. A Gamma model with weights 1.5 and(1 | gr(obs, dist = "student"))fitted to a logLik of 4.7e7 with a shape of 8e19 and two warnings. It is refused now, as the same model with(1 | obs)is, and the refusal names the term as written.
Other changes
frmtmb’s R CMD check (four platforms) and its coverage job fail when a test skips because a suggested package is missing or cannot be loaded. The extension jobs have done so since 0.74.0. Every job with this gate also fails when testthat’s output has no summary line. No package is allowed to be absent. The GEV density test reads extraDistr, a suggested package, in place of evd, which frmtmb does not declare. So it runs on every platform. Through 0.74.0 it skipped on all four R CMD check jobs and in coverage.
The pkgdown job installs every extension. It also runs the saved-fit test of
cox(),car()andsar()with neither extension loadable, and fails if any part of it skips.
frmtmb 0.74.0
The main changes:
- Four features moved to extension packages. Install the package and load it to use them:
-
cox()andcox_baseline():library(frmtmb.survival). -
royston_parmar()andrp_floored(), from the retired frmtmb.spline:library(frmtmb.survival). -
mixture_mvn():library(frmtmb.latent). -
car()andspde():library(frmtmb.spatial).
family = "cox",brmsfamily("cox"),car()andspde()stop with a message that names the package. The README shows how to install every extension at once. -
- frmtmb.spline is retired. Curve inference,
frm_curve(),frm_curve_deriv()andfrm_curve_feature(), is in frmtmb now. Remove an installed copy withremove.packages("frmtmb.spline"): it still loads with frmtmb 0.74.0, masks these three functions and replaces their print methods. - New seams for extension packages:
frmtmb_register_covstruct(),frmtmb_family(simplex_pars =, aterm_data =), the family hookpost$curve_check,us_sigma()andus_theta(), andfrmtmb_local_registries(). - Every brms family, addition term and formula special resolves in core, resolves in a named extension, or has a refusal that says why.
- Update the extensions together with frmtmb. frmtmb.latent 0.7.0, frmtmb.survival 0.1.0, frmtmb.spatial 0.1.0 and frmtmb.sample 0.21.1 need frmtmb 0.74.0, and frmtmb.latent 0.6.2 does not load with frmtmb 0.74.0.
Breaking changes
cox()andcox_baseline()are in the new extension package frmtmb.survival. Load it withlibrary(frmtmb.survival). brms hascoxin its core, sofamily = "cox",brmsfamily("cox")andfamily = cox()resolve as before with frmtmb.survival loaded. Without it,family = "cox"andbrmsfamily("cox")stop with “The cox family is in the extension package frmtmb.survival, which is not loaded”. A fit is the same fit. The tests have eightcox()models: censored, a frailty,df = 6with the identity link, truncated, asbhazprior, and the three spellings. On each, the log likelihood, the estimates and the standard errors areidentical()to frmtmb 0.73.0’s, andget_prior()gives the same table. Acox()fit saved by frmtmb 0.73.0 or earlier must be refitted: frmtmb refuses it by name and says to refit it with frmtmb.survival loaded. Everycoxpair offrm_compat()keeps its status and note.frm_curve(),frm_curve_deriv()andfrm_curve_feature()are in frmtmb now, with their print methods and the article “How to report a fitted curve”. They were in frmtmb.spline, which is retired. Load nothing extra: they apply to any fitted curve, not only to splines. On the curves, derivatives, features and difference curves the frmtmb.spline tests read, their output isidentical()to frmtmb.spline 0.10.2’s. Theirprint()methods now refuse an argument they do not take, as every frmtmb print method does; before, one was silently ignored.set_prior()no longer refuses adirichlet()on a class that is not one of frmtmb’s own, such asclass = "sigma", when the row is written: such a name can be a simplex that a family declares, which only the fit can tell.frm()refuses the row instead, and names the simplex classes, asset_prior()did.validate_prior()refuses it too. Adirichlet()on one of frmtmb’s own classes, such asclass = "b", is still refused byset_prior().mixture_mvn()moved to the extension package frmtmb.latent. To fit a multivariate normal mixture, install frmtmb.latent and runlibrary(frmtmb.latent). Without it,mixture_mvn()is an unknown function. The family, its arguments and its fits did not change. On the 59 fits of its tests, these are identical to the last bit on the same machine: the log-likelihood, the estimates, the standard errors,predict(),fitted(),mixture_probs()andsimulate(). Its rows offrm_compat()come with frmtmb.latent, and with it loaded the table holds the same rows as before. brms has no multivariate normal mixture, so no brms spelling reached the family, andfamily = "mixture_mvn"was not a supported spelling before the move either.mixture(),mixture_probs()andpp_mixture()stay in core. mclust is no longer in Suggests. Update frmtmb.latent together with frmtmb: an older frmtmb.latent does not load with this frmtmb, because itshmm()rules name themixture_mvnfeature that core no longer has.car()andspde()are in the new extension package frmtmb.spatial. Load it withlibrary(frmtmb.spatial). Without the package,frm()refuses a formula that writes either term and names the package. The fields are the same. On the models of the moved tests, these are identical to frmtmb 0.73.0’s: the log-likelihood, every estimate and standard error, the fitted values with and without an unseen location, theranef()conditional standard deviations,confint_varcorr()andsimulate().frmtmb.sample::frm_sample()gives the same draws from the same seed. With the package loaded,frm_compat()gives every pair the status it had. fmesher is no longer among frmtmb’s suggested packages.
New features
frmtmb_family(simplex_pars =)declares which extra parameters of a family are simplexes, by brms name:c(sbhaz = "sbhaz_raw")is the baseline of frmtmb.survival’scox(). A declared simplex is read in the chart of amo()simplex, and searched off a pole and off a face the likelihood rises from. frmtmb.sample draws it in the softmax chart under brms’sdirichlet(1), and names its draws as brms does (sbhaz[k]). It has a prior class of its own name inget_prior()andset_prior(). Until now these behaviors were written forcox()by name.frm()refuses, by name, a declaration that names no extra parameter the family returns, or one with no coordinate. Before, such a typo ended in “not enough degrees of freedom to define contrasts”.Some saved fits have a family that an extension package built. When this session cannot load that package, such a fit now stops at the first method that needs the family’s own functions. These include
summary(),vcov(),confint(),fitted(),residuals(),simulate(),update()andfrm_curve(). The message names the package: “The cox family of this model is from the extension package frmtmb.survival, which is not installed …”. It stopped with “could not find function” and the name of one of that package’s internals. A family records the package that built it, and amixture()the packages of its components.logLik(),fixef()andfrm_linpred(type = "link")need no function of the family and still answer.frmtmb_family(aterm_data =)builds data the densities read from the observed response, once, at frame assembly, such as a spline basis over the observed times.A family’s
post$curve_check(fit, resp)is run byfrm_curve()and its two companions before they read a curve, and may refuse. It is how frmtmb.survival’sroyston_parmar()refuses a curve off a fit whose cumulative hazard is not monotone.us_sigma(theta, d)and its inverseus_theta(S)are exported.us_sigma()builds the covariance matrix that the parameter vector of aus()block encodes. It is AD-safe, so a family in another package can carry a free covariance matrix as an extra parameter.us_theta()gives the parameter vector of a covariance matrix, for a start. The correlation entries fill the lower triangle by column, which is glmmTMB’s order up to 3 x 3 and not from 4 x 4.us_sigma()refuses athetaof the wrong length and names the length it needs; inside frmtmb a wrong length was recycled with a warning.floored = TRUEnow floors the variance of a 1 x 1 block as the 1 x 1 random-effect blocks do, which no caller in frmtmb reached.frmtmb_register_covstruct()lets a package register a covariance structure from its.onLoad(): a bar structure writtenname(terms | g), or a field written as a predictor special such ascar(M, gr = g). Its precision or covariance may depend on parameters and on fixed data that the special names. One entry carries everything that frmtmb and frmtmb.sample read: the objective, the standard errors and their checks,VarCorr(),confint_varcorr(),ranef(),predict()with and without new levels,simulate(),conditional_effects()andvcov_cluster(). The built-in structures are entries of the same shape. The structure joinsfrm_compat()as a feature of kind"covstruct", where quadrature andimportance =refuse it, asfrm()does.?frmtmb-extension-apihas the steps. The call checks the entry it builds, and each block is checked against its entry when the frame is built: the length ofstart, the positions ofsd_idx, the dimension and names ofvcov, and for a registered structure thatlpdfis finite at zero. The call refuses a name that the rest of frmtmb, brms or another extension owns, such asgaussianordec. It also refuses a function that formulas call as a term, such aspoly,nsorlog. A name that another package registered is replaced only withreplace = TRUE. A fit records the package each structure came from: read where that package is not installed, the fit is refused by the package’s name, with classfrmtmb_extension_missing, before a method reads the structure.frmtmb_local_registries()saves every registry a package fills from its.onLoad()and restores it when a test ends, for an extension’s own test suite.frm_compat()resolves the rules that packages contribute in the order of the packages’ names. So the status and note it gives a pair no longer depend on the order of thelibrary()calls.A brms spelling that an extension supplies is refused by name when that extension is not loaded, as the Wiener family already was:
y | dec(rt) ~ xnames frmtmb.eam and says how to load it. The brms names frmtmb does not support are refused with the reason and the frmtmb spelling:bhaz()(the basis is set incox(df =, degree =, intercept =)of frmtmb.survival, andbhaz(gr = )has no equivalent) andbrmsfamily("custom")(usecustom_family()). Every family, addition term and formula special of brms 2.23.0 now resolves in core, resolves in a named extension, or has such a refusal. This holds for its plainest spelling and for the others that brms documents. Before,custom,bhaz(),dec()and baregr(g)did not.
Bug fixes
A call around a bar term that names no covariance structure is refused, at any depth and in
re_formula =too. Through 0.73.0 the call was dropped and the term fitted with the defaultusstructure with no message:foo(1 | g),foo(bar(1 | g))andus_t(1 | g)all gave the log-likelihood of(1 | g), andre_formula = ~ foo(1 | g)was ignored.(1 | gr(g)), brms’s plainest spelling of a grouping factor, is(1 | g), as it is in brms. Through 0.73.0 it was refused.A gaussian model can have a field with one row per level, such as
car()over areas with one observation each. Such a model no longer warns that its variance is confounded with the residual sd. A fixed matrix correlates the levels, as forgr(cov = A), which the check already exempted; through 0.73.0 it named three structures and missed the rest.
Other changes
- gratia is a suggested package, for the curve tests that compare
frm_curve()’s simultaneous band with gratia’s. splines2 is no longer suggested: only thecox()tests used it, and they moved with it.
frmtmb 0.73.1
Bug fixes
-
fitted()of amultinomial()ordirichlet_multinomial()fit holds the interval of each expected count at the row’s trials. Before, the upper bound was the sum of the estimates, which equals the trials only to rounding, so a bound could pass them by a few units in the last place.
frmtmb 0.73.0
The main changes:
-
cens()andtrunc()for 23 more families, every one brms 2.23.0 censors, with brms’s reading of a discrete bound. -
brmsfamily("negbinomial2")anddirichlet_multinomial(). - The generics of brms’s
loo_*()functions,kfold(),reloo(),loo_moment_match()andloo_subsample(), with a refusal on a fit. -
quadrature = TRUErefuses a row that reads more than one random-effect block, andfrm()refuses a weighted likelihood with no maximum.
Breaking changes
-
A discrete response is censored and truncated as brms 2.23.0 reads its bounds. Right censoring at
kisP(Y > k), left censoringP(Y <= k), an intervalP(k < Y <= k2),trunc(lb = k)P(Y >= k)throughF(k - 1), andtrunc(lb, ub)the windowF(ub) - F(lb - 1). Through 0.72.0 a censoring bound on a count was inclusive (P(Y >= k)for right censoring), so the same data frame now means a different model: 200 poisson draws atlambda = 4recorded as “6 or more” and right censored at 6 move by 30.2 in log-likelihood and 5.5 percent inlambda. Right censored at 5, they give the old fit exactly. This applies to every discrete family,poisson()anddiscrete_weibull()included. The once-per-session notice about the inclusive reading is gone.- One composition stays frmtmb’s, on every response: a censored row under
trunc()is the event inside the window,P(y < Y <= ub)for right censoring andP(lb <= Y <= y)for left, over the window’s mass. brms 2.23.0 divides the event over the whole line,P(Y > y)orP(Y <= y), by the window’s mass, which is not a probability: on a poisson and a negative binomial withtrunc(lb = 2, ub = 9)and censored rows the two log-likelihoods differ by 6.16 and 14.85. They agree for right censoring with no upper bound, left censoring with no lower bound, and every interval. -
trunc(lb = 0), or anylb <= 0on a count, is no truncation and is accepted, as in brms; it was refused. - An interval whose ends agree (
y2 == y) is refused on a count, as on a continuous response:(y, y]holds nothing. A fractionaltrunc()bound on a count is refused, as a fractionalcens()bound already was.
- One composition stays frmtmb’s, on every response: a censored row under
trunc()refuses a response outside its window, and a window that is empty, with brms’s two messages: “Some responses are outside of the truncation bounds” and “Truncation bounds are invalid: lb >= ub”. Such a fit used to run, and scored a row the truncated distribution gives no mass to.The refusal of
cens()andtrunc()names the family. It points atfrm_compat()for the term that was used, which lists the families that have a CDF. It no longer lists them itself.The predictive draws of
dirichlet()andbrmsfamily("dirichlet2")change for the same seed. The Dirichlet draw that they anddirichlet_multinomial()share takes its gamma variates on the log scale. On the probability scale every variate of a row underflowed to 0 at a small concentration, and the row came outNaN: 18 rows of 20000 atphi = 0.01and 9570 atphi = 1e-3, none now.simulate(),predict()and the draws methods of a dirichlet_multinomial fit with a smallphistopped with “NA in probability vector”. The distribution of the draws is unchanged; the means and covariances match the Dirichlet’s within Monte Carlo error atphifrom 0.01 to 20. brms 2.23.0’srdirichlet()has the same defect.quadrature = TRUErefuses a row that reads more than one random-effect block. Nested blocks,(1 | ga/gb), crossed blocks,(1 | g) + (1 | h), and(1 | g)in bothmuandsigmaused to fit. But the Gauss-Kronrod rule integrated only some of the effects and left the rest to the Laplace approximation. Over(1 | ga/gb)the log-likelihood equaled the Laplace one to 1e-8, andsummary()failed with “no ‘dimnames’ attribute for array”. They are now refused before any tape, as lme4 refusesnAGQ > 1for more than one term, and the message names the blocks. One block per row is unchanged, including the blocks of a by-splitgr(g, by = f).frm()refuses a weighted likelihood that has no maximum. The refusal applies when two conditions hold. First, every row with a positive weight has a random effect of its own inmu. Second, the weights sum to more than the number of rows. If the dispersion formula has an intercept, the likelihood grows without bound as the dispersion goes to its limit (sigmato 0,shapeto infinity). It grows like the peak height of the density to the powersum(w) - n. Such a fit used to stop at “NA/NaN gradient evaluation” underprofile = TRUEandREML = TRUE. Under ML it reported a log-likelihood of 1.4e11 with only convergence warnings. The refusal names the term as written and the weight sum, and names what gives a usable fit: weights that sum to the number of rows or fewer, a random effect that is not observation-level, or a fixed dispersion. It covers gaussian, student, lognormal, Gamma, weibull, Beta, inverse.gaussian, skew_normal, shifted_lognormal, huber, von_mises, frechet, gen_extreme_value, exgaussian and zero_inflated_asym_laplace. A prior exempts the model only when it provably bounds the dispersion.gamma()onshapedoes, and so does anormal()prior on every coefficient of an intercept-free cell-means formula (sigma ~ 0 + f). Astudent_t()orgamma(2, 1)onsigmadoes not, and neither does a prior on the slopes of a dispersion formula with an intercept. A prior that bounds the objective does not make the estimate informative: the mode then sits where the prior and the weight excess balance. A dispersion formula that cannot reach its limit at every row (sigma ~ 0 + x, or an offset alone) is not refused: its likelihood is bounded. The check does not run withse(), residual correlation,mi()or several responses. Undercens()it counts the uncensored rows, because a censored row’s probability is at most 1 and its own effect can bring it to 1. Under the Laplace approximation each censored row takes 1/2 off the excess.asym_laplace() with a random effect of its own on every row is refused under the Laplace approximation, at any weights. Its log density is linear on each side of the mode, so the data give such an effect no curvature. The approximate likelihood grew without bound: log-likelihood 449 with
sigmaat exp(-7.1) on 80 unweighted rows, where the exact integral gives -56.7.quadrature = TRUEintegrates the effect exactly and is refused only when the weights sum to more than the number of rows.zero_inflated_asym_laplace()is refused the same way: on its nonzero rows it is asym_laplace (log-likelihood 278.05 on 80 unweighted rows). Withcens()both checks count the uncensored rows.
New features
The generics of brms’s
loo_epred(),loo_linpred(),loo_predict(),loo_predictive_interval(),loo_R2(),loo_model_weights(),kfold(),reloo(),loo_moment_match()andloo_subsample(), with a refusal on a fit.log_lik(),loo()andbayes_R2()refuse a fit the same way. On a fit the sixloo_*()functions namefrmtmb.sample::frm_sample().kfold(),reloo()andloo_moment_match()namefrm_bootstrap()andAIC(), because neither frmtmb nor frmtmb.sample refits the sampler, andloo_subsample()names the fullloo()of draws. Before, with frmtmb alone a call stopped at “could not find function”, and with rstantools, loo or brms attached at “no applicable method”.loo_moment_match(fit)with loo attached stopped inside loo’s default method, at a missing argumentloo. frmtmb.sample registers its draws methods on these generics. Each defers to its owner’s generic while the owner is loaded (rstantools; loo forloo_model_weights(),kfold(),loo_moment_match()andloo_subsample(); brms forreloo()).loo_epred()has two owners, brms 2.23.0 and rstantools 2.7.1, and takes the one nearer the front of the search path.-
cens()andtrunc()take 23 more families, the ones brms 2.23.0 censors and truncates:student(),skew_normal(),exgaussian(),shifted_lognormal(),asym_laplace(),zero_inflated_asym_laplace(),von_mises(),Gamma(),Beta(),binomial(),beta_binomial(),negbinomial(),geometric()andbrmsfamily("com_poisson"). The others are the zero-inflated poisson, negative binomial, binomial, beta-binomial and beta, and the hurdle poisson, negative binomial, gamma and lognormal. Each family has both tails of its CDF on the log scale on the AD tape, and a censored model with random effects fits. Each tail is computed directly where it is small, so a right-censored row deep in the upper tail keeps its digits and its gradient. Three are the complement of the other tail in part of the range:com_poisson’s survivor above1e-3, a hurdle family’s CDF where the survivor is below 1/2, and von Mises tails on the near side of the mean. The references arestats, the closed forms, 300-bitRmpfrarithmetic and adaptive quadrature, over grids that include both tails and the edges of the support. Against them, the relative error of the log of either tail is 1e-12 or better on 554 of 582 comparisons. It is 3.7e-9 or better on the rest, except two negative-binomial far-tail points wherestats::pnbinom()is the one in error. The incomplete gamma’s is 8.04e-14 or better for shapes from 0.01 to 1e7.- A censored row is brms’s likelihood. At frmtmb’s estimates, the log density of brms’s Stan program with flat priors equals
logLik(), given the same data frame. This holds for every family with all four censoring codes, and for truncated fits. Among those are a zero-inflated count under a constant bound and a right-censored count insidetrunc(lb = ). The gradient is not asserted forasym_laplace(), whose fit stops at nlminb’s false convergence on these data with or without censoring. Two exceptions are brms’s. Its inverse-gaussian survivor adds a term it should subtract: -0.466 where the survivor is -0.988 aty = 2,mu = 2,shape = 4. So frmtmb’s right-censored inverse.gaussian fit differs from brms’s. Also, Stan’s gradient of the gamma CDF in the shape is -0.18 where finite differences of its own log density give 0.0002. That was measured at frmtmb’s estimates of one censored hurdle_gamma model.compois()takes the mean where brms’scom_poisson()takes the rate, so the two are different models. -
von_mises()follows Stan and brms:F(y) = P(-pi <= Y <= y). -
bernoulli(),xbeta()andzero_one_inflated_beta()still refuse both terms, as brms does, andfrm_compat()says so. - A count right-censored at or above its trials (
binomial(),beta_binomial()and their zero-inflated forms), or an interval that starts there, is refused by name:P(Y > n)is zero. brms accepts such a row, its log density is-Inf, and it draws nothing.
- A censored row is brms’s likelihood. At frmtmb’s estimates, the log density of brms’s Stan program with flat priors equals
A family can give both tails of its CDF on the log scale, through the new
cdf_logargument offrmtmb_family(). For such a family every censored row and everytrunc()window is formed on the log scale from the tail that keeps its digits: an interval asS(y) - S(y2)whereS(y) >= 1/2and asF(y2) - F(y)below. So an interval in an upper tail keeps its digits, and a left-censored row whose probability is below1e-308keeps its log. Every built-in family with a CDF declares it exceptexponential(),weibull(),cox(),frechet(),gen_extreme_value()anddiscrete_weibull(), which are read throughlcdfandlccdfas before. For every familytrunc(lb = )with no upper bound divides by the log survivor atlb, as brms does.fitted()reports the truncated mean of atrunc()response forstudent(),Gamma(),binomial(),negbinomial()andgeometric(), the families brms has one for. A count’s lower bound is inclusive, as in the likelihood. brms 2.23.0 sums fromlb + 1, the mean of a different window than its own likelihood uses: for a Poisson mean of 3 truncated to[2, 6]it reports 3.9588 where the truncated distribution’s mean is 3.3869.dirichlet_multinomial(), brms’s overdispersed multinomial. The category probabilities of each row are Dirichlet with meansoftmax(mu)and precisionphi, and the counts are multinomial given them: brms’sdirichlet_multinomial_logit2_lpmf(). The response,trials(),refcat,Kandfitted()aremultinomial()’s, and so is the code: the two families are one implementation.phitakes a log link (link_phi) and its own formula.simulate()draws each row’s probabilities from the Dirichlet, then the counts. On 126 rows (two to five categories,phifrom 1e-4 to 1e8) the log density is within 9.1e-12 of an exact reference. brms 2.23.0’s Stan function is within 6.1e-7: its lgamma differences lose digits asphigrows, and these are formed so that they do not. That matters becausephigrows without bound on data with no overdispersion: on 7 such fits the plain differences stopped above the multinomial limit, 3 with “false convergence”; formed this way, none does. Fit to the same data as brms, with and without a group term, the maximum-likelihood estimates lie within 0.05 posterior sd of brms’s posterior means. This holds for the coefficients andphi.brmsfamily("negbinomial2"), brms’s negative binomial insigma = 1 / shape. The variance ismu + sigma * mu^2. It isnegbinomial()underneath: the same density, simulator, mean, variance, deviance and addition terms, read atshape = 1 / sigma, so anythingnegbinomial()gains this family gains with it.sigmatakes the positive links (link_sigma) and its own formula; with the log link on both,sigma ~ zis the modelshape ~ zwith its coefficients negated. As in brms there is no exported constructor. Over 384 points withsigmafrom 1e-8 to 1e6, its log density is within 6e-12 relative of an exact reference. brms 2.23.0’sneg_binomial_2_log_lpmf()is within 1.4e-10.Both families are in
frm_compat(), reach every post-fit path ofnegbinomial()andmultinomial(), and take brms’s link sets.
Bug fixes
simulate()of atrunc()fit stopped on a valid model when a row kept only a few percent of its fitted mass inside the window: rejection sampling left 1 of 120 rows of a weibull fit with adiag()term unfilled after 100 passes. A row that rejection has not filled after 20 passes is now drawn by inverting its CDF, for each family of this package with one.trunc(lb = )without an upper bound divided by1 - F(lb)on the probability scale, which is 0 onceF(lb)rounds to one: an infinite likelihood the optimizer could step into. A gaussian fit with its mean far below the bound had an objective of-Inf. A weibulltrunc(lb = 0.5)fit with adiag()term stopped at a NaN gradient, and now fits at a log-likelihood of -145.97.An interval-censored row lost its digits above the median and at a point mass: as
log(F2 - F1)both terms round toward one in an upper tail, and a hurdle’s(0, y2]was(hu + tiny) - hu. The objective was then rounding noise. At points away from the optimum its gradient was 21 and 45 percent off forskew_normal()andvon_mises(), and eight times its size forhurdle_gamma().trunc(lb = -Inf)ortrunc(ub = Inf)stopped on a NaN gradient fromInf * 0in the derivative of the normalizer. An infinite bound is now no bound.The
exgaussian()density was a gaussian approximation fromsigma / betaof about 20, the oneRTMBdist::dexgauss()uses: on 400 gaussian rows the log-likelihood was 0.63 off atsigma / beta = 50, and near-gaussian fits stopped atsigma / beta = 20with false convergence, below the maximum. The density is now the closed form, through Mills’s ratio, and so is the CDF, whose derivatives were wrong fromsigma / betaof about 1e3.inverse.gaussian()’s CDF wasNaNonceshape / mupassed 355, whereexp(2 shape / mu)overflows, so a censored or truncated fit there had aNaNlikelihood. Both tails are now formed in logs: atmu = 1,shape = 1000the CDF at 0.9 is 4.534060403e-04, asstatmod::pinvgauss()gives.poisson()andinverse.gaussian()score a right-censored row with the log survivor itself. It waslog(1 - F), which is-InfonceFrounds to one. A Poisson row right-censored at 700 with mean 5 now scores -2773.28. A Poisson fit right censored at 75000, which failed at-Inf, reaches the maximum of an independent fit onstats::ppois().
frmtmb 0.72.0
The main changes:
- brms’s
categorical()andmultinomial()signatures,refcat,fitted()andposterior_linpred()of every location predictor, andlogistic_normal()inmvbf(). - Per-coefficient
sdpriors, brms’s labels and order of nested and multi-membership groups, and one Student-t latent scale for the terms of one grouping factor. -
uniform()andlognormal()priors, and support guards on the tape. - Every addition term on
newdata, coded as the fit coded it. - Accurate standard errors for Cook’s distance, D1 and D2, REML in the design’s coordinates, and bounded Newton steps.
- The exact Hessian for every fit without random effects.
-
drop1(),step(),update()andinfluence()refit on the data the fit holds.
Breaking changes
multinomial()takes brms’s arguments,(link, refcat, K), and reads its categories from the response matrix.multinomial()with no arguments now fits, as brms spells it; the dpars are named by the columns,cbind(y1, y2, y3)givingmuy2andmuy3, as in brms.multinomial(K = 3)still builds the family away from the data, with the positional namesmu2,mu3.multinomial(3)is refused: the first argument is the link, and the refusal namesK =.A
categorical()dpar name is brms’smake_stan_names(). Dots and underscores in a level are dropped, so the level"p.q"givesmupqwhere it gavemup.q.fitted(scale = "linear")on a family with several location predictors returns brms’sn x 4 x Parray. That is everymu<cat>ofcategorical(),multinomial()and the simplex families, layerseta1, eta2, ..., and every component’smu<k>of a mixture, layers unnamed. It returned the first predictor’sn x 4matrix.categorical()takesrefcatas its second argument, as brms does;levelsandKfollow it.Nested and multi-membership groups carry brms’s labels.
(x | g1/g2)is(x | g1) + (x | g1:g2), as brms 2.23.0 expands it. The groups areg1andg1:g2. reformulas wroteg2:g1. The group of(x | mm(m1, m2))ismmm1m2, whatever its weights. These labels are inset_prior(group = ),default_prior(),prior_summary(),variables()(sd_g1:g2__x),ranef(),coef(),VarCorr(),ngrps(),summary(),re_formulaand the new-level messages. The old spellings"g2:g1"and"mm(m1, m2)"are refused, as brms refuses them. Are_formulaterm now names its group exactly as the fit writes it.The groups of each predictor are in brms’s order, sorted by name (brms’s
terms_re()). This order is invariables(),ranef(),VarCorr(),ngrps(),summary()and the order of thethetaparameters.(1 | h) + (1 | g)now listsgfirst. Terms on one group keep their written order.ranef()andcoef()name the levels of an interaction group as brms does. The parts are joined with_:p1_q1for a level ofg1:g2(wasp1:q1). A:inside a part stays:c:d_f.variables()lists every group-level standard deviation, then every correlation, then a smooth’ssds_, in brms’s order. Before, each block’s correlations followed its own standard deviations.-
Student-t terms of one grouping factor share one latent scale per level, as in brms. Examples are `(1 | gr(g, dist = “student”)) + (0
- x | gr(g, dist = “student”))
, and a term inmubeside a term insigma. At each level, their coefficients are one multivariate t. So the first example is the model(1 + x || gr(g, dist = “student”)). Agr(g, by = f)term beside a plain term ongshares the scale at each level ofg. Agr(g, by = f)term alone is the model it was. Terms that share a scale take onedist_nu.simulate(re_formula = )keeps or redraws them together, andquadrature = TRUErefuses them. See?frmtmb-student-re`.
- x | gr(g, dist = “student”))
A prior density with a bounded support is
NaNoutside it on the tape too.beta(),gamma(),inv_gamma(),exponential()andlognormal()each add a guard. The guard isNaNoutside the support and exactly 0 on it, its edges included as in Stan.beta()andgamma()relied on alog()that the tape drops where it is multiplied by a constant 0. Sobeta(1, 1)on a coefficient was flat over the whole line, andgamma(1, r)was finite below 0.exponential()had no guard at all and was finite below 0 on doubles too. Inside the support nothing changes.A bounded density on a parameter with no such bound of its own warns, in brms’s words. Examples are
uniform()orbeta()on a coefficient andgamma()on an intercept. It warns when neither its row nor the class row above it giveslborub, as brms 2.23.0 does. The warning comes from the call that uses the prior (frm(),validate_prior(),update(),frmtmb.sample::frm_sample()). Without the bound the parameter is not constrained, and the density is a wall a sampler meets: givelbandub, whichfrm_sample()passes to Stan as a transform.-
predict(newdata =)reads every addition term from newdata, and a binomial-type family refuses a missingtrials(). Before, the predictive draws read onlytrunc(),rate()andthres(gr = )from newdata, and the simulator was given no other term. A binomial withy | trials(N)and newdataN = 1000drew 0 or 1 on every draw (column mean 0.555;simulate()andfitted()gave 563). A gaussian withse(s)drew at sd 0.029 where newdata hads = 5. A custom family’svint()andvreal()and frmtmb.eam’sdec()failed. The defect is in every release from 0.61.0, whenpredict()became a predictive summary. Nowpredict()reads the terms assimulate()andfitted()read them: binomial gives 563.3 and 1.08 (brms 2.23.0: 562.7 and 1.15), andse()gives sd 0.059 and 4.77 (brms: 0.061 and 4.95). The same fix reaches the newdata fill of acov = FALSEARMA term, andmultinomial()predict(newdata =), which was refused, now draws newdata’s trials.What changes for a caller: newdata without the
trials()orse()variable is refused, naming the term, as brms refuses it. Before, it drew one trial per row or ignored the known standard error. The refusal ends “supply the variable”; it used to offertype = "conditional", whichpredict()does not take. A binomial-type family no longer reads a missing
trials()as one trial.binomial(),zero_inflated_binomial(),beta_binomial()andzero_inflated_beta_binomial()refuse it by name. A model of these families always has the term, so only a caller that drops it meets the refusal.A custom family’s declared
required_atermsare checked after the fit too.sim_response()andresponse_mean()refuse a term list that lacks one, naming the term. Before, the family’s own code metNULL, which gave “invalid arguments” fromrbinom()or an empty answer.An addition term coded by its levels is read on newdata through the fit’s codes. This is a factor for any term, and a factor, character or logical value of a registered term such as frmtmb.eam’s
dec(). Before, newdata was coded by its own levels. A releveled or characterdec()swapped the two boundaries in silence. One value alone (a one-row newdata, or a logicaldec()givenTRUE) was refused for holding one level. A value the fit did not see is now refused by name. It used to be coded as if it were one of the fit’s. A coercion’s own refusal is reported in its own words, not as “could not be evaluated on newdata”.conditional_effects()holds the variables ofthres(),vint()and registered terms atround(median()), as brms holds its whole-valued terms, and says so in a message. They were held at the mean: avint()count of 9.985 drewNA, and a numericdec()at 0.573 was no boundary.The
conditional_effects()grid refuses a row without itsthres(gr = )group (method = "predict"), where it used to draw the row without it. The default grid holds the group, so the refusal meets only a grid that cannot evaluate it.-
REML and
profile = TRUEfits on an ill-conditioned fixed design are taped in the design’s coordinates. Therebetais integrated with the group effects. Its inner Hessian has the design’s condition number squared, and the REML objective lost its digits.- On a raw polynomial of degree 7 with
(1 | g), the fit stopped short with a convergence warning. Its coefficient standard errors were 1.5 to 1.9 times the exact ones. At degree 8 every standard error was NaN, or the optimizer failed. - Now the objective is taped in
gamma, withbeta = T gammaandTfrom the design’s QR. Under REML the tape also carries-log |det T|, sologLik(), AIC and BIC are those ofbeta. - At degrees 3 to 7 the REML log-likelihood equals the exact restricted log-likelihood to 6.6e-10. At degrees 5 to 8 the fit reaches the REML optimum to 2e-12. The covariance is within 6.1e-9 of the exact one on the scale of the correlation.
- The joint covariance is held as a factor, as on an ML fit. Every route that propagates it reads the factor.
vcov()warns (classfrmtmb_cov_products) where a product of the whole matrix loses its digits, andmarginaleffects::get_vcov()returns NaN. - frmtmb’s REML
vcov()adds the uncertainty of the variance parameters to the conditional covariance, as glmmTMB does. lme4’svcov()is the conditional part alone. On these designs it is 2.2 to 3.3 percent off its own exact value at degree 7. At degree 8 lme4 stops with an error.
- On a raw polynomial of degree 7 with
-
A log standard deviation that the optimizer leaves near zero below a small interior maximum now moves. The log-likelihood is flat toward zero in a log sd. So nlminb can stop at an sd near zero and call the stop converged.
- Take a raw polynomial of degree 7 with
(1 | g)and no group effect (seed 4). The stop was at an sd of 1.9e-6. The maximum is at 0.033 times sigma, 1.2e-3 higher. The fit warned, and every coefficient lost its standard error. - When a larger sd gains more than
grad_tolwith the rest held, the optimizer restarts from it. That fit now reaches the exact maximum to 8e-9. Its covariance is within 5.6e-6 of the exact one. - Among 80 such fits, three were more than
grad_tolshort. All three now reach the maximum. One of them had called the sd “flat”. - A caller who asks for an earlier stop keeps it. That is an
iter.maxoreval.maxbelow 1000, a looserrel.tol,x.tol,abs.tolorxf.tolthan nlminb’s, or another optimizer. Higher caps and tighter tolerances keep the restart.
- Take a raw polynomial of degree 7 with
-
Newton steps from the stop respect the bounds a prior sets. Every full step that crossed a bound was refused. A step now holds the coordinates it would push out at their bounds.
- Take an upper bound one standard error below the estimate of a raw polynomial’s top coefficient. On 0.71.0, 10 of 20 fits reached the constrained maximum at degrees 5 to 8. Two of the others stopped 0.12 and 0.60 short with no warning. Their standard errors were 5.8 to 39 times the right ones. Now 17 of the 20 reach it.
- A parameter on its bound that the Newton step would push out is held by that bound, whatever the sign of its gradient. On an ill-conditioned design that sign is rounding.
- The other coefficients’ covariance is taken in the coordinates of the design without the held one.
-
Nonlinear formulas take the coordinates of their Jacobian. A nonlinear body has no design, but its Jacobian with respect to the coefficients of its parameters serves as one. This changes the standard errors of such fits.
- A degree-8 raw polynomial written as a body, one parameter per coefficient, now reaches the maximum of
lm(). Its covariance is within 4.7e-6 of the exact one. 0.71.0 stopped 0.047 to 0.41 short and gave no standard errors. - A degree-6 raw polynomial beside
A * sin(w * z)had a covariance up to 3e-3 from the exact one, with no warning. Now it is within 1.8e-8. - The Hessian in these coordinates must be symmetric to 1e-3 of its largest entry. A body that is nearly unidentified along a nonlinear parameter fails that test and keeps the covariance of 0.71.0.
- A degree-8 raw polynomial written as a body, one parameter per coefficient, now reaches the maximum of
-
Fits with up to 400 outer parameters take the design’s coordinates. The limit was 100. This changes the standard errors of such fits.
- A degree-7 raw polynomial beside 150 or 292 covariates now reaches the maximum of
lm(). Its standard errors are within 5.3e-6. On 0.71.0 these fits stopped 0.5 to 2.2 short, with a warning, and their standard errors were NaN or wrong. - Newton steps in natural units still stop at 100 parameters.
- A degree-7 raw polynomial beside 150 or 292 covariates now reaches the maximum of
-
On an identity-link gaussian design the Hessian in the design’s coordinates costs half the gradients. There the gradient is linear, so a central difference is exact. Every other column keeps the Richardson extrapolation. This moves such fits’ standard errors in their last digits.
- With 100 outer parameters and n = 1e4, a gaussian fit takes 0.87 of the time it took on 0.71.0. With
(1 | g)it takes 0.89 of that time. - With n = 1e5, both fits take 0.88 of the time they took on 0.71.0.
- Non-gaussian and nonlinear fits keep the cost and the accuracy of 0.71.0.
- With 100 outer parameters and n = 1e4, a gaussian fit takes 0.87 of the time it took on 0.71.0. With
Standard errors of a fit without random effects come from the exact Hessian. The covariance was the inverse of central differences of the gradient with an absolute step of 1e-3, the step
stats::optimHess()takes. Where that step moves a linear predictor by much and the gradient is not linear along it, the standard errors were wrong and nothing warned. A poisson GLM with x ~ N(500, 100) had standard errors 6.2 percent off. Take ten poisson and bernoulli designs with covariates of very different scales, against an exact covariance formed in 256 bits. The error was up to 21 percent. It is now below 1e-11 on 18 of the 20 fits, and 2.6e-8 on the other two (a covariate on [1000, 1001]).y ~ a * exp(b * x)with x on 20 + [0, 1] had a correlation 0.133 off, and is now within 5e-7. The covariance is now the inverse of the AD Hessian, which the standard-error check already read as exact. Every fit without random effects moves. On 27 ordinary, well-scaled fits the standard errors moved by at most 5.2e-7 relative, and the correlations by at most 1.1e-6. All moved toward the exact values. The fit already holds this Hessian, so the firstvcov()no longer takes one: on a bernoulli GLM with 50 coefficients and 1e5 rows it went from 20.4 s to 2.2 s. Underfrmtmb_control(check_se = "ignore")the fit does not hold it, and the firstvcov()builds it. That is faster than the differences, but it takes more memory. On a bernoulli GLM with 1e4 rows the peak memory rose by 350 MB at 200 coefficients and by 870 MB at 500. The change includesquadrature = TRUEandimportance =fits, whose outer objective has no inner random effects, and there it is the largest. Take a poissony ~ x + (1 | g)fit by importance sampling with x ~ N(500, 100). Its intercept standard error was 0.498. The same model on x / 100, an exact reparameterization, gives 0.270, so the first was 85 percent off. It now gives 0.270 on both. The quadrature fit of the same model was 0.27 percent off.With random effects, the outer Hessian’s step follows the design. Where the root mean square of a coefficient’s design column is above one, the step is 1e-3 divided by it. So a step moves the linear predictor by about 1e-3 in root mean square over the rows. A row far out in a heavy tail moves by more. Take a lognormal column with a root mean square of 50 and a largest value of 836: that row moves by 1.7e-2. The standard errors are within 7e-6 of a Richardson reference, where 0.71.0 was 1.6e-2 off. A poisson GLMM with x ~ N(500, 100) and
(1 | g)had standard errors 0.37 percent off and a correlation 0.10 off. Its group standard deviation lost its standard error as “flat”, with a warning. It now keeps it, and the covariance is within 5.3e-7 of a Richardson reference. A design whose columns are all at unit scale or below keeps the steps of 0.71.0, so its covariance does not change. A nonlinear parameter, amo()term and a variance parameter keep the step of 1e-3.A variance that is zero, negative or not finite is shown as
NaN, never as itself.asym_laplace()’s log density is piecewise linear in its location, so its exact Hessian has no curvature there. Inverting it gave standard errors of exactly 0 andconfint()intervals of zero width. Such a parameter now has aNaNrow and column invcov(), andsummary()anddiagnose()give the reason. Noasym_laplace()fit has a valid Hessian standard error, in this version or in 0.71.0, whose differences across the kinks gave noise.update()takes what it does not replace from the fit, as brms’supdate()does: the data, the formula, the family, the control, the start and the flags. Sinceupdate()was added (0.6.0) it evaluated the symbols of the stored call again. So the refit used the objects of that name at that time. Withx2changed in the workspace,update(fit)gave a log-likelihood of -3597.28 for a fit of -3536.91. With 10 rows deleted it had 1990 observations for the fit’s 2000. It stopped with “object ‘d’ not found” (or'form','fam') in three cases. The data were removed, the fit was read back withreadRDS()in a new session, or it was made inside a function. A globalformof the same name as the call’s argument became the model without a word. The data the fit holds are the model frame and the variables the formula reads through a transform. They are on the rows the fit used, afterna.action. The fit’s bases go with them. Soupdate(fit)with nothing changed gives the fit’s log-likelihood, also whenna.actionremoved rows and the formula hassplines::ns(),poly()orscale(). Rows thatna.actionremoved stay out of the refit, and underna.excludethe refit pads its residuals to the fit’s rows. The new fit’s call names everything as the stored call did.A variable the fit does not hold is read only from data that are still the fit’s.
update(fit, ~ . + z)with a newzreads the stored call’s data, asstats::update()andadd1()need, when those data hold the fit’s rows and values. Otherwise it is refused with brms’s words, “New variables found: ‘z’ Please supply your data again via argument ‘newdata’.”, and a reason. It used to read every variable again from the changed object.drop1(), and sostep(), andanova()refuse a fit whose prior adds a log density to the objective. ItslogLik()is the log-likelihood plus the log prior (since MAP fits, 0.10.0), so the difference of two is not a likelihood ratio: on one probe it was -1.56, with p = 1. A prior that only sets bounds adds no density and is not refused. The message points to a refit without the prior, or to frmtmb.sample’sloo()andbayes_factor(). brms has nodrop1()and compares models with those two.drop1()keeps the fit’s bases. A submodel keeps the bases that the fit computed from every row beforena.action: the knots ofns(), the coefficients ofpoly(), and the center and scale ofscale(). It keeps their coefficient names too, asstats::drop1()keeps the fit’s design columns. With three missing values in another variable, the likelihood-ratio statistics now equalstats::drop1()’s onlm()andglm(). The AIC of a submodel equals that ofupdate(fit, ~ . - x3).influence()keeps the fit’s bases in every deletion refit, as lme4’sinfluence.merMod()does. Onz ~ poly(x2, 2) + x + (1 | g)the deleted-group coefficients differed from lme4’s by 1.3 times the largest displacement. Each deletion computed the basis again from its own rows (since 0.15.0). They now agree to 4.6e-6 of it.A brms
categorical(refcat = "b")object passed asfamily =keeps itsrefcat. 0.71.0 dropped it and fitted referenceaunder the namesmubandmuc. The same call now fits brms’s parameterization, with brms’s names.validate_prior()andprior_summary()list the class"sd"rows in brms’s order: the class-wide rows, then group by group, with the responses inside each group.conditional_effects(categorical = TRUE)oncategorical()draws a Wald band fromfitted()by default, where it asked forband = "boot".When the standard-error check removes nothing, a log sd with a small curvature is interior. The covariance is then taken in the design’s coordinates, which moves the standard errors of such fits. Before,
solve()of the natural Hessian kept it 1.4e-3 from the exact one at degree 7.
New features
categorical(refcat = )andmultinomial(refcat = )name the reference category, as in brms:NULLis the first, a name must be one of the categories, andNAgives every category a predictor. The dpar names,default_prior()rows and post-fit shapes follow the reference.brmsfamily("categorical", refcat = )andbrmsfamily("multinomial", refcat = )take it too. Underrefcat = NAa maximum-likelihood fit determines only the category probabilities. Its standard-error warning names the common shift of every category’s predictor. A prior on each predictor determines the coefficients.logLik(),AIC(),BIC(),anova()anddrop1()count the degrees of freedom of the same model with a reference category, which has the same likelihood. With a category formula of its own the count is not defined: the df isNA, andanova()anddrop1()say why. The same holds fordirichlet(refcat = NA). brms 2.23.0’s own fit of that model with its default priors does not converge (R-hat 1.71). A family’sprint()shows its reference category.fitted()onmultinomial()returns brms’sn x 4 x Karray of expected counts, layersP(Y = <category>), with delta-method standard errors; 0.71.0 refused it.frm_linpred(type = "response")gives then x Kcounts.logistic_normal()fits insidemvbf(). Its correlations are a block of its response, set withset_prior(class = "lncor", resp = ). The summary reports them aslncor_<resp>(b,c), andvariables()andhypothesis()aslncor_<resp>__b__c.<resp>is brms’s Stan name of the response (YaforY_a). brms 2.23.0 cannot compile this model; its model block, repaired, agrees with frmtmb’s likelihood at frmtmb’s estimates.dirichlet()inmvbf()worked and is now tested against brms.A class
"sd"prior can name one coefficient, as in brms.set_prior("normal(0, 0.5)", class = "sd", group = "g", coef = "x")puts the density onsd_g__xalone. This works infrm(),validate_prior()andfrmtmb.sample::frm_sample().default_prior()lists a row per coefficient below each group row, in brms’s order. A brms prior table with such a row translates. Acoefwithout agroupis refused by name. So is acoefon a structure whose columns share one standard deviation (homdiag(),ar1(),homcs()). See “Group-level standard deviations” in?set_prior.gr(g, id = )is brms’s correlation id.(1 | gr(g, id = "a")) + (0 + x | gr(g, id = "a"))is(1 | a | g) + (0 + x | a | g).re_formula = ~ diag(x | g)names the term it wraps, as~ (x | g)does. It named no term before.A refused class
"cor"prior names a||orgr(cor = FALSE)term as it was written,(1 + f || g), not as1 + f | g [diag].set_prior()takes brms’suniform(lb, ub)andlognormal(mu, sd). frmtmb.sample needs both for brms’s defaults on a non-decision time without a formula and on an ordinaldiscunder its identity link. The uniform density isNaNoutside its interval. That rejects a value there; it does not keep a sampler inside, whichlbandubdo.A natural-scale prior on a logit-linked distributional parameter (
set_prior("beta(1, 1)", class = "zi"), and frmtmb.sample’s defaults) has a finite Jacobian for every value of its coefficient. Past a coefficient of about 37 it was minus infinity, so a mode there gave an infinite objective.A prior on the
nuthat the Student-t responses of aset_rescor(TRUE)model share takes noresp, as brms’s row has none:set_prior("gamma(2, 0.1)", class = "nu"). It was refused with and withoutresp.The sampling API exports
ord_lp_blocks(), the threshold blocks of an ordinal model with thedparclass"Intercept"addresses each by.-
cooks.distance()and the Wald rules ofanova()onfrm_multiple()fits read the factor of the covariance. These are D1, and D2 withuse = "wald". On 0.71.0 they inverted the whole matrix.- At degrees 7 and 8 they stopped with “system is computationally singular”. At degree 6 Cook’s distances were up to 5.8e-4 off.
- Now they are within 2.3e-6 of the exact ones at degree 8.
- Where the factor’s rows are dependent to working precision, they give NaN with a warning.
Bug fixes
cumulative()andhurdle_cumulative()gave a log probability ofNaNon doubles for an end category whose neighboring thresholds are tied. The category has a positive probability there. This held under every link with a log-odds form (logit, probit, probit_approx, cloglog). Fits were not affected, because the tape drops the masked term;log_lik()of draws was. The taped value and gradient are unchanged bit for bit at distinct thresholds. Ahurdle_cumulative()zero row at such a tie wasNaNtoo, and islog(hu)now.conditional_effects()onmultinomial()draws one curve per category, as brms does withcategorical = TRUE, and as for the simplex families.dharma_residuals()on it is refused by name.set_prior(resp = )takes brms’s Stan name of a response (Yafor a responseY_a) as well as frmtmb’s own.confint(parm = )takes the name of a correlation that is one internal parameter:rescor__y1__y2of two responses andlncor__b__cof a three-categorylogistic_normal(). The refusal for a wider block advisedhypothesis(method = "profile"), which fails there (“not linear”); it now nameshypothesis(class = NULL)andmethod = "boot".drop1()oncategorical(),multinomial(), the simplex families and mixtures was refused (“No linear predictor for dpar ‘mu’”). A dropped term now leaves every location predictor that reads the main formula.A
mixture()draws each component at its own rows’ addition terms. The simulator gave each component the whole term list. So the j-th row of a class read thetrials()of the j-th row of the data. Take a binomial mixture withNof 2 and 50. Insimulate(),predict()and frmtmb.sample’sposterior_predict(), 25% of the draws exceededN, and theN = 2rows averaged 11.8 wherefitted()says 0.98.mixture(groups =)had the same defect (21%). Present in every release with mixtures of trials families.predict(newdata =)andpredict(re_formula =)hand the simulator each distributional parameter on its natural scale. They handed it the reporting scale, which for a mixture’sthetais already the softmax probability, and the simulator applied the softmax again: the mixing weights were wrong on every draw away from the in-sample fast path (443 againstfitted()484 on a binomial mixture atN = 1000). The defect is in every release from 0.61.0 (862032e3), where these draws first read the dpars that way.conditional_effects(method = "predict")codes a registered addition term, such as frmtmb.eam’sdec(), with the coercion its package registered, andthres(gr = )as the fit’s group codes. Adec()factor reached the simulator as its level codes, so each level was drawn at the other boundary.drop1()refits a term that reads a raw variable. These stopped with “The model usesx2, which is not a column ofdata” (since 0.49.0):z ~ x + log(x2),scale(x),I(x^2),splines::ns(x2, 3),poly(x, 2)beside another term,x * log(x2), a transformed response, anddrop1(scope = ). The refits read the model frame, which holdslog(x2)and notx2. The likelihood-ratio statistics and degrees of freedom equal those ofstats::drop1()onlm()andglm()and of lme4’sdrop1()onlmer(REML = FALSE). brms has nodrop1().drop1()refits a model with an intervalcens(cc, y2). The upper bound is not a column of the model frame, so every refit stopped (since 0.49.0).drop1()takes the family and the control from the fit. It evaluated them again from the formula’s environment, sofamily = famfrom a function’s frame was “object ‘fam’ not found” (since 0.49.0).step()removes a term, on the fit’s data. It passes its update toupdate(evaluate = FALSE)as text,"~ . - x3", whichupdate()did not read. So every step that removed a term stopped with “object ‘change’ not found” (since 0.21.0, when drop1() was added). The call it evaluates now holds the fit’s data and formula by value. Sostep()gives the same model when the data object is changed or removed, or in a new session afterreadRDS(). Its path equalsstats::step()’s onlm()andglm().influence()reads the raw variables by default. Its default data, the model frame, made every deletion refit of a model with a transformed term fail (since 0.15.0).fitted(fit, re_formula = ~ (1 + log(x2) | g)), the fit’s own term, was refused as matching no group-level term (since 0.61.0).make_conditions(fit, "x2")refusedx2oflog(x2)(since 0.65.0). Both read the model frame.A fit on a tibble with a transformed term no longer warns “Setting row names on a tibble is deprecated”. A fit on a named list keeps the raw variables a refit reads. Both were defects since 0.66.0.
Documentation
-
?vcov.frmtmb_fitsays what the REML covariance of the coefficients is: the conditional part plus what the variance parameters’ uncertainty adds. It names how that differs from lme4’s.
Internal
- Bin 2 of the ported brms suite reaches 0.71.0’s families. The thin adapter maps brms’s per-category predictors onto
categorical(),dirichlet(),brmsfamily("dirichlet2"),logistic_normal()andmultinomial(). It shims brms’s Frechet, generalized extreme value and categorical functions onto frmtmb’s families, and reaches frmtmb.eam’swiener(). Some of brms’s checks read only the shape of a log density or a CDF. A value check now compares such a value with brms’s own function on the same arguments. No package code changed.
frmtmb 0.71.0
Seven lanes, each with an adversarial review and fix rounds:
-
dblbar: every||term read as brms reads it. -
famsimplex: brms’s simplex families,dirichlet()andlogistic_normal(). -
famuni:frechet(),gen_extreme_value()anddiscrete_weibull(), and frmtmb.eam’swiener()links. -
postfix: the post-processing gaps of the brms port. These arefrm_multiple()methods,pp_check(resp = ), and the default-start error of a nonlinear fit. -
optstart: least-squares starting values for location-scale fits, and a restart that neither gains nor moves. -
illse: accurate standard errors on ill-conditioned designs. -
remeasure: the vignette port and bin 2 of brms’s suite, measured, with no package code.
Each lane’s dev/<lane>-findings.md has the validation, the numbers and the scripts, and dev/reviews/2026-10-13-<lane>.md has the review. dev/round-20261014.md is the round’s record.
Breaking changes
Every
(x || g)term is read as brms reads it. It is(x | gr(g, cor = FALSE)): one term, coded with the formula’s own contrasts. Each column has its own standard deviation, and there is no correlation.A factor in a
||term has the columns of the|term.(1 + f || g)with three levels offnow hassd_g__Intercept,sd_g__fbandsd_g__fc. It had lme4’s column for every level (fa,fbandfcbeside the intercept). That is one more standard deviation than brms has, and a different model.(1 + z + f || g)had 5 standard deviations and now has brms’s 4. Fits of a factor in a||term change. This reverses the earlier reading that(f || g)gives a column per level. lme4 still expands||term by term (lme4#818); frmtmb follows brms.The random-effect design of 29
||spellings is brms 2.23.0’s, column for column (standata()). Thesd_names are brms’s, and there is nocor_. The spellings cover numeric and factor terms, interactions,poly(),gr(by =),g1:g2,g1/g2andmm(). They also cover|ID|terms beside||terms,sigma, nonlinear parameters,mo(),cmc = FALSEandmvbf(). Their sd rows ofdefault_prior()are brms’s too. The exception is the group label of a nested or multi-membership term, which differs for|terms too.A
||term is one block.(1 + x || g)was two blocks,1 | gand0 + x | g. It is now one block with independent standard deviations, as in brms. Soranef(fit)$gandVarCorr(fit)$gaddress it by the factor name. For gaussian group-level effects the likelihood is the same. The estimates of a numeric(1 + x || g)agree with the old fit to 1.5e-12 relative.A
||term overgr(g, dist = "student")has one latent scale per level, shared by all its columns, as in brms. The old split gave each piece its own scale, which is a different model. So these fits change, numeric terms included. On one data set,(1 + x || gr(g, dist = "student"))moved from logLik -538.612 to -539.121.An uncorrelated block names no correlation. brms has no
cor_parameter for such a block. frmtmb named one, fixed at zero, for each pair of columns of adiag()orhomdiag()block. It also did so for thediag_tblock of a||term overgr(dist = "student").variables(),summary(),confint_varcorr()andhypothesis()list none now. This includeshypothesis()on frmtmb.sample’s draws.A constant on a location of a family with several locations is refused. This covers
categorical(),multinomial(),mixture()and the new simplex families. It also covers the extension families with several locations: frmtmb.eam’slba()andrdm()(v<k>), and frmtmb.latent’shmm()(mu<k>) andlca()(theta<k>).bf(f ~ x, muc = 0)andbf(y ~ x, mu2 = 0)used to fit, and the location was estimated anyway. The constant had no effect, so the fit was not the model the user wrote. brms 2.23.0 ignores the constant the same way.The refusal gives two routes.
muc ~ 1estimates the location.muc ~ 0 + offset(o), withoa data column, holds it at a known value.fitted()andresiduals()of a fit accept brms’sndraws. In brms it sets how many posterior draws the summary averages. That is the Monte Carlo error of the answer, not the quantity answered. A maximum-likelihood fit answers that quantity at its estimate, with no Monte Carlo error. Sondrawschanges nothing, and a message says so; it was refused.draw_ids,sort,summary = FALSEandrobust = TRUEask for particular draws and stay refused. The setup lines of brms’s ported rowsbrmsfit-methods:314and:317now run (dev/postfix-findings.md, item f).-
A location-scale fit starts
muat least squares and its scale at the spread thatmuleaves. Before,mustarted at an intercept read off the response, and the scale started at the response’s own spread (sd(y)for gaussian()). Where a covariate explains most of the response, that spread is far larger than the residual one, and a fit could stop short in silence. For example, takey = 1 + 1e4 z1 + b_g + ewith n = 101000 and 50 columns. That fit stopped 9107.66 log-likelihood units below lme4 at code 0. Its group sd was at its boundary.Now the coefficients of
mustart at a least-squares fit on its fixed design. The fit is weighted when the response hasweights(), is on the link scale, and takes the offset out. The scale starts at that fit’s residuals, in the family’s own convention:- sigma: gaussian(), lognormal() and shifted_lognormal() and hurdle_lognormal() (on the log scale), student() (from the median absolute deviation at
nu = 4), skew_normal(), asym_laplace(), zero_inflated_asym_laplace() and huber(); - sigma and beta: exgaussian(), each at half the residual spread, as the family’s initializer gives each half of
sd(y); - shape: Gamma(link = “log”), weibull() and negbinomial();
- precision: Beta().
The rule does not apply to a family with no scale or not in that list. It does not apply to another link of
mu, or to an intercept-only or nonlinearmu. It does not apply to amuwith a smooth or agp()term. It does not apply to a design whose condition number is above 1e5, or to a mixture. It does not apply to a censored, truncated orse()response, or to a scale held at a constant. There,muand the scale start as before, bit for bit. The group sds and the units can still differ from 0.70.0’s, because the units of anmi()response and of zero_inflated_asym_laplace() changed (below). Astartor a prior does not turn the rule off. The entriesstartnames are the user’s, and the others keep these defaults. The function par_template() shows the start.The change was measured on 550 fits of 11 families (
dev/optstart-findings.md). Fits short of the maximum went from 108 to 69. Warned fits went from 129 to 91. On high-R^2 and offset designs, student(), skew_normal() and exgaussian() stopped short on 8 to 10 of 10 fits each. They stopped up to 1370 log-likelihood units short. Now none of them stops short. Other changes that a user can see:- The
gr(g, by = f)fixture of test-se-check.R now reaches its boundary, 1.44e-4 higher. It warns “singular convergence (7)”. 0.70.0 stopped short there, and warned that two standard errors are not available. - Fit a negative binomial to Poisson data, and the shape’s maximum is at infinity. Such a fit ends at “false convergence (8)” at the maximum on 4 of 10 seeds. On 0.70.0 it did so on 2 of 10.
- A gaussian fit capped by
iter.maxno longer stops short: from the least-squares start it is at its maximum already. - exgaussian() with a group sd 30 times sigma still ends at a NaN gradient on 5 of 10 fits. That is as on 0.70.0, and 4 of the 5 seeds are the same. From the true values, with the group effects started at 0, the density fails on 10 of 10. From the true values and the true group effects, it fails on 3 of 10.
- sigma: gaussian(), lognormal() and shifted_lognormal() and hurdle_lognormal() (on the log scale), student() (from the median absolute deviation at
A restart from the optimum no longer turns a converged fit into a warning. Where the first run converged, the restart from it may stop at a nonzero code without moving. The fit then keeps the first run’s code, iterations and evaluations, if two things hold. The restart’s gain is within the optimizer’s relative function tolerance of the objective. And no coordinate moved by more than a relative step tolerance. The function tolerance is nlminb’s
rel.tol, or L-BFGS-B’sfactrtimes machine epsilon foroptimizer = "optim". The step tolerance is nlminb’sx.tol(1.5e-8 by default); foroptimizer = "optim"it is 1.5e-8. The restart’s own code, message, iterations and evaluations are kept infit$opt$restart, and diagnose() shows them. Before, the restart’s “false convergence (8)” replaced the converged code and warned. This happened at lme4’s maximum on 2 of 20 fits with a group sd 1000 times the residual sd. It also happened on brms_missings’fit_imp2.The unit of zero_inflated_asym_laplace() is measured on its nonzero rows. A weighted response’s unit is measured on the residuals of the weighted least-squares fit. An
mi()response’s unit leaves out its imputed rows.-
Standard errors on an ill-conditioned fixed design. A raw polynomial of high degree, a covariate far from zero or a near-collinear pair can make the outer Hessian too ill-conditioned. Then it cannot be inverted in the coefficients’ own coordinates. frmtmb now takes the Hessian by differences of the gradient along new coordinates. These are the columns of the inverse of each design’s QR factor. In these coordinates the Hessian is close to the identity. The covariance is then mapped back exactly, as
lm()does through its QR. This applies when the Hessian has a condition number above 1e8 at unit diagonal. Its weak directions must also lie in the coefficients of the linear predictors.- The designs measured: raw polynomials of degree 5 to 8 on [1, 2]. The families are gaussian with and without
(1 | g), bernoulli, poisson and cumulative. Every entry ofvcov(full = TRUE)is within 2e-5 of the exact covariance at the fit’s own point. That is on the scale of the correlation, against a 256-bit reference.fitted()standard errors agree with the exact ones to four digits. Where a reference usespoly(), they agree with it to four digits too. - On the same designs, 0.70.0 gave no standard error to the nine coefficients of a degree-8 polynomial (“the design is ill-conditioned”). With
(1 | g)under bernoulli, poisson and cumulative it lost three to six of them as “flat”. At degrees 6 and 7 it was up to 20 percent off. It gave the category probabilities of a cumulative fit standard errors up to 270000 times the true ones. It gave no warning. - These fits keep the verdicts of 0.70.0. A design that its QR finds rank deficient is one; the frame drops such a column, as
lm()does at degree 9. A nonlinear formula is another. So are a fit that stopped short of the maximum, and a fit with more than 100 outer parameters. - Some parameters outside the designs have almost no curvature. A group standard deviation at zero is an example. The SE check judges such a parameter as on 0.70.0: “boundary”, “short of the maximum” or “not finite”. When the check removes it, the covariance of the other parameters is taken in the design’s coordinates. That covariance is conditional on the removed parameter.
- The test set: raw polynomials of degree 5 to 8 with
(1 | g)and no group effect, 20 seeds each. The sd gets the verdict of 0.70.0 on 77 of the 80 fits. On the 16 degree-8 fits with the sd at zero, the coefficients get the standard errors oflm()to 3e-5. 0.70.0 gave those 16 fits no standard error for any coefficient. - The extra cost is about two Hessians in the design’s coordinates, at 4 p gradients each. One is for a step from the stop. The other is for the standard errors at the maximum. With n = 1e5 and 100 parameters, a fit took 1.45 to 1.66 times as long as on 0.70.0 in two runs. With
(1 | g)it took 1.47 to 1.52 times as long. On 0.70.0 these fits stopped short of the maximum. Now they reach it.
- The designs measured: raw polynomials of degree 5 to 8 on [1, 2]. The families are gaussian with and without
-
Every route that propagates the covariance reads it through a factor on such a fit. No double-precision matrix can be multiplied by a row of that design without losing every digit. At degree 8,
a' V afrom the exact covariance rounded to double is 2.8 times the true variance of a fitted value.- These routes use the factor:
fitted(),predict()and its parameter draws,frm_linpred(), and the ordinal and categorical category probabilities. So doconditional_effects(),conditional_smooths(),hypothesis(),VarCorr()and the pooled hypotheses offrm_multiple(). emmeans now takes the grid route on these fits, which also uses it. -
frm_lp_basis()returns the factor asV_rootandV_rest, withV = V_rest + V_root V_root'.frm_joint_cov()returns it asrootandVrest. -
vcov_cluster()forms the sandwich through the factor. It returns the factor of the sandwich as the attributeroot, andhypothesis(vcov = )reads it. On a degree-8 fit with(1 | g), the sandwich from the whole bread was off by 2.3 to 12 on the scale of the correlation. A fitted value’s standard error through it was 0 to 1511 times the exact one. Through the factor every entry is within 2e-5, and that standard error is within 1e-6. - These routes read the whole matrix: a product that a caller forms from
vcov()orvcov_cluster(), car and lmtest. They are not accurate on such a fit.cooks.distance()and the D1 and D2 tests ofanova()onfrm_multiple()fits invert the whole matrix. They can stop with an error. - A product formed from the whole matrix can keep fewer than about four digits. That happens at a condition number above 1e12. Then
vcov(),vcov_cluster()andfrm_joint_cov()warn once per fit (classfrmtmb_cov_products). The warning names the routes that are right. - marginaleffects forms
J V J'itself fromget_vcov(), withJfrom finite differences. On such a fit the cancellation in that product multiplies the error ofJ. On a binomial with mixed scales (condition number 4e11),predictions()standard errors were 4.8 to 15.1 times the exact ones, with no warning. Soget_vcov()now returns NaN, with a warning, on every fit whose covariance is held as a factor. - ggeffects, modelbased and parameters read
insight::get_varcov(). It returns NaN, with a warning, where a product loses its digits. -
hypothesis(vcov = )can get a matrix that does not carry its factor. Then each hypothesis that combines parameters gets NaN, with a warning.
- These routes use the factor:
-
Newton steps from the stop and the gradient verdict use the same coordinates. On these designs the natural Hessian has no Cholesky factor. So a degree-8 fit stopped up to 1.2 below the maximum. Its gradient warning measured the remaining headroom from a matrix with no digits along the weakest direction. Both now use the design’s coordinates, and the degree-8 fits reach the maximum of
lm().- A log standard deviation whose Hessian row is empty, such as a group sd at zero, cannot show its headroom through the Hessian. So the headroom asks the likelihood what larger values of that sd gain, the rest held. That gain is added to the headroom of the other parameters, and only then compared with
grad_tol.
- A log standard deviation whose Hessian row is empty, such as a group sd at zero, cannot show its headroom through the Hessian. So the headroom asks the likelihood what larger values of that sd gain, the rest held. That gain is added to the headroom of the other parameters, and only then compared with
-
The standard-error check judges a fit at its maximum whatever nlminb’s code. It skipped every fit with a convergence warning. It showed the standard errors of
sdreport()unchecked. So on a ridge, rounding decided whether a fit’s standard errors were judged. Take the xbeta fit of lane ci691 from 25 starts a few ulps apart. The 8 fits with code 0 lost all three standard errors as flat. The 17 fits at “false convergence” showed finite ones.- The “did not report convergence” warning now says what a Newton step from the stop gains. Within
grad_tol, the estimates are at the maximum. The check judges them as it judges a converged fit, with the same messages. 23 of the 25 xbeta fits now lose the three standard errors. The two that keep them stopped where the curvature is above the check’s threshold, whatever their code. - Above
grad_tol, the warning says that the estimates and standard errors are those of a point short of the maximum. The check does not judge them, because a curvature verdict there is not about the data. - The eight test suites have 129 fits that nlminb did not call converged. A restart from the stop showed 104 of them at the maximum.
- Newton steps in the design’s coordinates also start from such a stop. Take
y ~ xwith x on [1e5, 1e5 + 1]. It stopped at “false convergence” 157 below the log-likelihood oflm(). Now it reaches it.
- The “did not report convergence” warning now says what a Newton step from the stop gains. Within
New features
gr(g, cor = FALSE)is read as in brms: it is the||term.mm(g1, g2, cor = FALSE)is the||term over the membership grouping.cor = TRUEis the default. Under||, acorargument is ignored, as brms ignores it.gr(cor = )andmm(cor = )were refused.dirichlet()andlogistic_normal(), brms’s simplex families. The response is ann x Kmatrix of proportions. Write it ascbind(y1, y2, y3)or as a matrix column of the data. Each row must sum to one, and the column names are the categories.The first category is the reference unless
refcatnames another, as in brms. Each other categorykgets its own predictormu<k>. The main formula applies to every category.bf(Y ~ x, muc ~ z)gives one category its own formula.(1 | ID | g)correlates the group effects across categories.dirichlet()has the precisionphi.logistic_normal()has onesigma<k>per category, and each takes a formula. It also has one correlation matrix,lncor. The summary reports it aslncor(b,c), and the new prior class"lncor"takes anlkj()prior.The log densities agree with brms 2.23.0’s Stan functions to a relative 1.5e-11 on 7200 grid points. brms’s Stan program at frmtmb’s estimates gives the same log density to 6e-13 on six models. 500 simulated fits recover their parameters with no warning (
dev/famsimplex-findings.md).brmsfamily("dirichlet2")reaches brms’s unexported Dirichlet family, as in brms. It has one log-link predictor per category for that category’s concentration.fitted()on adirichlet()fit returns brms’sn x 4 x Karray of category means, with layers namedP(Y = k).predict(),simulate(),posterior_predict()andlog_lik()work on both families. Their draws are rows on the simplex.conditional_effects()draws one curve per category, with a Wald band fromfitted(). As in brms,fitted(),conditional_effects()andposterior_epred()refuselogistic_normal(), whose mean has no closed form.residuals(),dharma_residuals()andpp_check()refuse both families.The response check holds each row to 1.5e-8 of one. brms tests the MEAN departure over the rows, so one row off the simplex passes there in a large data set.
dirichlet()then fails at sampling, andlogistic_normal()fits a row that is not a composition.-
Three brms families that
frm()refused:frechet(),gen_extreme_value()anddiscrete_weibull(), with brms 2.23.0’s parameters, links, link options and defaults (see?frmtmb-extreme-families). They take random effects, distributional formulas,weights(),cens()andtrunc(), and they can be components of amixture(), as in brms.fitted(),predict(),simulate(),residuals()(includingtype = "pearson"and"osa"),conditional_effects(),pp_check()anddefault_prior()cover them. Their log densities agree with a 256-bit reference to about 1e-13 on a grid that includes the far tails. They agree with brms’s Stan functions where those are accurate. brms’s are not everywhere. Its Frechet log survivor is off by up to 3.8e-3 and is-Infin the far right tail. Its GEV density loses digits nearxi = 0, 6.8e-8 atxi = 1e-7(dev/famuni-findings.md).-
frechet():muis the mean, as in brms, and the scale ismu / gamma(1 - 1 / nu). Underlink_nu = "identity"anuat or below 1 has no likelihood. -
gen_extreme_value(): the support depends on the parameters, and the density is 0 outside it, with a log that falls steeply rather than aNaN. The optimizer is turned back, and in amixture()a component that does not reach a row adds nothing to it. brms’s density isNaNthere.xi = 0is the Gumbel model, exactly, sobf(y ~ x, xi = 0)fits one.fitted()returnsInfwhere the mean is infinite (xi >= 1). brms’sposterior_epred()returnsNaNatxi = 0andxi = 1and a finite, wrong number above 1. - An
xiwithout a formula takes thelog1plink, which brms uses for a predictedxi. brms scales such anxiinto the support the data leave. That is the same maximum, but its range reaches belowxi = -1, where the likelihood is unbounded. Underlink_xi = "identity",xi <= -1has no likelihood, so every link holdsxiabove -1. A fit that stops within 1e-3 of -1 with the likelihood not stationary inxithere warns that its log-likelihood is a supremum. A maximum inside that band gets a message instead. -
discrete_weibull():fitted()is the mean to about 1e-12, where brms’sposterior_epred()stops the series at the first term below 0.001.cens()follows frmtmb’s inclusive discrete convention. So right and interval censoring differ from brms, which reads a right bound asP(Y > y). Subtract one from those bounds to reproduce a brms fit. Left censoring andtrunc()agree with brms.
-
family = "wiener"(or a brmswiener()object) takes the family from frmtmb.eam when that package is loaded. Otherwise it is refused with a message that names frmtmb.eam. Before, it was refused as an unsupported family, with a list of the core families. A brms object keeps its links, sobrms::wiener()fits brms’s log link forndt(see?frmtmb.eam::wiener).For frmtmb.sample, a sampling chart for a scalar GEV
xi(gev_xi_chart_frame()and its companions,?frmtmb-gev-xi-chart), so thatfrm_sample()samples it through brms’stmp_xi.A
frm_multiple()result hasvcov(),fitted(),coef(),ranef(),VarCorr(),nobs(),formula(),family()and an emmeans basis. Each pools the imputations by Rubin’s rules, where brms’sbrm_multiple()combines their draws. The test case is a poisson model on 5 mice imputations, againstglm()pooled bymice::pool(). The largest relative differences are 1.4e-5 infixef(), 8.5e-6 invcov()and 3.1e-5 in the pooled degrees of freedom (dev/postfix-validate.R).fitted()of afrm_multiple()result pools on the link scale, as itsconditional_effects()does, and transforms back. Its interval therefore stays inside the response’s range. On the poisson case it agrees withmice::pool.scalar()ofglm()‘s linear predictor to 7.5e-6. A category probability takes only its interval from the logit scale. Its estimate is the mean of the imputations’ probabilities, so the categories sum to 1.emmeans()of afrm_multiple()result is emmeans’s own pooling of amice::mira. It agrees with that pooling ofglm()fits to 4.4e-5 in the means and 4.3e-4 in the degrees of freedom.ranef()andcoef()of afrm_multiple()result average the modes and coefficients of each level.ranef(condVar = TRUE)gives Rubin’s total variance.VarCorr()pools on the log and Fisher-z scales of the pooled table, and its estimates match that table. Before,coef(),fitted()andresiduals()returnedNULLin silence (stats’ default methods).pp_check(resp = )checks one response of a multivariate fit. It compares that response’s observed values with its own simulated replicates, on its own rows, as brms does. Asubset()response uses the rows of its subset. A missing, plural or unknownrespis refused in brms 2.23.0’s words. Every multivariate fit was refused.The sampling API (
?frmtmb-sampling-api) exportsce_method(),ce_predict_family_check(),ce_sim_one()andpp_check_resp().ce_plan_eval()takesone =. frmtmb.sample reads them forconditional_effects(method = "predict")andpp_check(resp = )on draws.
Bug fixes
(x | p || g)and(x || p | g)are refused in brms’s words. The second was read as a grouping variablep.(x | p | gr(g, cor = FALSE))is refused too; brms drops the key there without a word.A known-covariance term with several columns and any structure other than the default is refused. This covers
gr(g, cov = A)andgr(g, prec = Q), and a factor in a||term over one of them. The known-covariance block correlates the columns of a term within a level, so such a term silently became a correlated block. A numeric(1 + x || gr(g, cov = A))is unchanged. So is a one-column term in any structure, such ashomdiag(0 + x | gr(g, cov = A)).The message about a duplicated group-level effect names a
||term as written, not as thediag()term it is rewritten to.A family with several locations takes a formula for the first one too.
bf(f ~ x, mub ~ z)on acategorical()response was refused whenbis the first predicted level. So wasbf(y ~ 1, mu1 ~ x, mu2 ~ x)on amixture(). brms 2.23.0 builds the mixture form and the same first-category formula ondirichlet(). Oncategorical(), brms stops with an internal error.A brms family object passed as
family =keeps itsrefcat.The pooled
conditional_effects()of an ordinalfrm_multiple()result has categories that sum to 1. Each estimate is the mean of the imputations’ probabilities; it was the inverse logit of their mean logit. The band still comes from the logit scale.The error of a nonlinear fit that fails from its default start names this model’s nonlinear parameters. It gives the two ways to give starts:
start =with the names ofpar_template(), or a located prior. It says that frmtmb offers no start of its own, because a searched start can converge at a local maximum with no warning. Its example namedult, a parameter of another model.A
frm_multiple()result refuses by name the methods that have no Rubin’s-rules meaning. They arepredict(),residuals(),pp_check(),logLik()(and soAIC()andBIC()),loo(),waic(),update()andmodel.frame(). Each says why, and namesanova()or one imputation’s fit,x$fits[[j]].frm_multiple()refuses imputations whose fits have different coefficients, and names them. It stopped insidevapply()with “values must be length 4”.summary(fit, waic = FALSE)is accepted. It asks for nothing, and brms 2.23.0 prints it as it printssummary(). The refusal ofwaic = TRUEstays, with its reason.frm(threshold = )is refused by name. The refusal names the family argument the threshold structure belongs to, asfamily = sratio(threshold = "equidistant"). It reads the family fromfamily =or frombf() + family(). brms before 2.0.0 read the argument inbrm(), and brms 2.23.0 refuses it too. It was R’s “unused argument”.The standard errors of an
mi()fit are right again. This is lane remeasure’s R1. On brms_missings’bf(bmi | mi() ~ age * mi(chl)) + bf(chl | mi() ~ age), 0.70.0 reported “false convergence (8)” at 0.67.0’s optimum, with every standard error NaN. Withmi(se)it withheld both residual-SD standard errors as flat and gave the rest up to 35% too large. There were two causes. First, the coefficient of ami()orme()term was stepped in the response’s units. Its unit is the response’s over the spread of the variable the term stands for. Second, the response units read an imputed value, stored as 0, as data. Now the standard errors of both fits agree with a closed-form reference to 2.1e-6 and 6.0e-7 relative. In that reference each row’s pair is bivariate normal, with the missing values integrated out.A finite-difference outer Hessian that does not factor is retaken along the coordinates whose step was too large. A step is too large when the objective rises by more than 1e-2 over it. Those columns are taken again at the step that gives a rise of 1e-4. The retake is kept when it factors and its smallest eigenvalue is above its own noise. A Hessian that factors is unchanged, bit for bit.
A
mo()simplex split by a category that no row is in loses the split’s standard errors on every fit at its maximum. The two steps around such a category enter only as their sum. That is an exact flat direction, curved in the simplex’s chart. The SE check read it off the Hessian. It missed it where the fit stopped a tolerance away from the floor of that ridge. The 0.70.0 consolidation had four designs and 160 fits. 8 of them kept zeta standard errors up to 949, with no warning. The split is now taken from the data, as a design alias is. None of the 160 fits is silent. 156 name the split. The other 4 stop short of the maximum with a large-gradient warning, as on 0.70.0.
Documentation
-
?brmsfamilyand?frmtmb-families(“Conway-Maxwell-Poisson counts”) say that themuofcompois(), whichbrmsfamily("com_poisson")gives, is the mean of the count. brms 2.23.0 fitsmuas the rate of the density, so the coefficients of the two fits are on different scales unlessnu = 1(dev/remeasure-findings.md, R6).
frmtmb 0.70.1
The macOS check of 0.70.0 failed where the Ubuntu and Windows ones passed. dev/ci701-findings.md has the causes and the numbers.
Bug fixes
confint_varcorr()keeps the sd intervals of a block where one parameter has no standard error. Before, the NaN row of that parameter went into every row of its block. So a correlation at +/-1, or one sd at zero, removed the other sd intervals.VarCorr()still reported their standard errors (8 of 20 fits ofy ~ x + (1 + x | g)with no slope variance). The delta method now reads the covariance asVarCorr()reads it. Only a row that moves with the lost parameter is NA.The standard-error check does not call identified standard deviations flat beside a correlation at +/-1. This applies to an unstructured block of dimension 2. Its correlation parameter can run past the clamp of
confint_varcorr()(a correlation of 0.9999). There the finite-difference Hessian is noise. On a(1 | q | id)block, a fit that stopped with code 0 at t = 2000 lost both sds as flat. Now the check first removes such a correlation, as a boundary, if the likelihood no longer changes along it. Then it takes the Hessian of the other parameters with the correlation held at the clamp. The sds then get the standard errors of the rank-one model that the boundary corresponds to, to 2e-4. The same rule runs on a code-1 fit thatse_boundary_stop()reads as a boundary stop.On other fits with such a correlation, the standard errors move toward an independent reference. On 6 of 20 fits of
y ~ x + (1 + x | g)with no slope variance, the largest relative error was 7.6e-4 to 5.3e-3 in 0.70.0, and is 1.2e-4 to 2.8e-4 now. The largest move is 0.55 percent.A test asked
confint_varcorr()to warn about a boundary. On some platforms the fit’s own boundary message names it instead. Another test did not allow “singular convergence (7)” on a degenerate ordinal mixture. nlminb reports it in 7 of 20 fits on ulp-moved data, at the same log-likelihood. Both tests now ask for what holds on every platform.
frmtmb 0.70.0
Three lanes, each with an adversarial review and punch rounds: moparity (brms features that were refused or diverged: mo() crossed with a factor and in group-level terms, Dirichlet priors on a simplex, update(prior =) merging as brms merges, the cox() baseline), optrap (fits that stopped short of the maximum because of the response’s units, a restart that raised, or a badly conditioned design) and seflags (which standard errors a fit reports: an exact design alias, curved ridges, an sd the likelihood does not read, raw polynomials). Each lane’s dev/<lane>-findings.md has the validation, the numbers and the scripts, and dev/reviews/2026-10-08-<lane>.md has the review; dev/round-20261013.md is the round’s record. In brms’s own ported suite, bin 1 passes 389 of 494 assertions (390 at 0.69.0): brmsfit-methods:326 asks a fixture that does not converge for finite standard errors, and it held at 0.69.0 by the luck of where the optimizer stopped.
Breaking changes
A factor, character or logical multiplier of
mo()is coded as brms codes it, where it was refused.mo(x) * fwithfof levelsa,b,cfitsmox,mox:fbandmox:fc, each with its own simplex and coefficient, andmo(x):ffitsmox:fa,mox:fbandmox:fc: the columns of brms’s special-term model matrix. A logical multiplier ismox:xTRUE, as in brms, where it wasmox:x. A label’s factors are named as brms names them, in the ordermodel.matrix()reads the special terms alone, soz + mo(x):zisz:mox(it wasmox:z) whilez * mo(x)staysmox:z, andmo(x) * z * wfits, withmox:z:w, where it was refused. brms 2.23.0 writes wrong Stan code for these terms (every column multiplied by the first covariate) and stops on a factor of three or more levels crossed with the main term (brms#1828); frmtmb fits the model brms 2.22.0 fits. Against brms’s compiled program for the same model written with numeric indicators, the log density agrees with frmtmb’s at its estimates (to 1e-6 relative, after brms’s flat Dirichlet constants), with brms’s gradient below 1e-3 there.update(prior = )merges the new prior into the stored one, as brms’supdate()does: a new specification replaces the stored one for its slot (class,coef,group,resp,dpar,nlpar), and the stored specifications for every other slot stay. It replaced the stored prior, so a storednormal(0, 0.05)onb_xwas lost byupdate(fit, prior = <a sigma prior>). The same holds for abrmspriorand for a table fromdefault_prior()or brms’sget_prior(), where every row takes its slot, as in brms: a row with an empty prior returns its slot to the default (flat), so an unedited row of a whole table resets the stored prior of its slot.prior = NULL, or a prior with no rows, keeps the stored prior, as in brms. A stored specification the updated model cannot take is still dropped with a message; a NEW one is still checked asfrm()checks it, where brms drops it silently.cox()holds its baseline simplex in the chart of amo()simplex, not a softmax, andfrmtmb_control(mo_search = )checks a weight at 0 too. Of 200 fits of a right-censored model, 5 stopped at “singular convergence (7)” with weights near 1e-50 and a warning, and one 1.86e-4 below the maximum; now 200 of 200 are within 1.8e-8 of the maximum, at code 0, with no warning, for 1.05 times the objective evaluations. Thesbhaz_rawcoordinates have other values;cox_baseline()reads the weights as before.A gaussian, student, lognormal or other location-scale fit reaches the maximum likelihood whatever the units of its response, and the fit of
y * kis the fit ofyscaled, to about 1e-7 in the log-likelihood. The identity-linkmuof these families now has a unit: the spread (the MAD) of the response after the fixed-effect design is taken out by least squares, on evenly spaced rows for a design of more than 5e6 cells, rounded to a power of two. The standard deviations of its random-effect terms start at that unit (they started at 1 in the response’s units), nlminb steps its coefficients in the unit over each column’s spread, and the inner solver’s gradient tolerance is divided by it.par_template()shows the new starts. Before, withyin units 1e3 times larger, 20 of 40 random-intercept fits stopped short of lme4’s maximum at optimizer code 0 and without a warning, by up to 0.67 log-likelihood units; withyin units 1e6 times larger, 40 of 40 sleepstudy-like random-slope fits, by up to 127, and 10 of 10 plainy ~ xfits, by up to 23. Now every one of these fits over units from 1e-6 to 1e6 is within 2.1e-7 of lme4’s orlm()’s maximum, at code 0. A response whose residual spread is between 0.71 and 1.41 has a unit of 1 and fits as before. Where it has a unit, the estimates agree withlm()and lme4 to about 1e-5 of a standard error (at most 8.5e-5 over 160y ~ x + zfits; before, 2e-7 to 1.2e-5 at moderate units and 2.8 standard errors withyin units 1e6 times larger), because nlminb’s stopping tests now read steps in the unit. Mixtures, extension families and asigmaheld at a constant (se()) keep their starts.An alias in a linear predictor’s design is tested exactly. A smooth’s fixed column joins the design after the check that drops aliased columns; on brms’s example fit 1 it is exactly
-0.046 + 0.190 * Age. The standard-error check found the alias only to the precision of the Hessian, so under OpenBLAS’s LAPACK all five predictions offitted(newdata)gotNaNand a warning though they are estimable, and a hypothesis 12 percent off the alias got a finite standard error with no warning. The check now takes such an alias from the design itself: every coefficient it moves (on fit 1 the intercept too) hasNaNas its standard error, and a prediction or hypothesis has one exactly when it does not move along the alias. The prediction test reads every direction in the unit-diagonal metric of the Hessian, so its verdict does not depend on the units the optimizer works in. On a mixture whose mixing weight has run to zero, a prediction that reads the vanished component’s flat parameters now getsNaNand the warning.Curved ridges lose their standard errors. A fit that stops within its tolerance of a ridge along which the likelihood is flat on a curve (
a + exp(b),a * b,c0 + exp(a)^kwith a group effect, amo()simplex with an unobserved category, two group terms on the same factor) has a small eigenvalue along the ridge that is not curvature of the likelihood, and the check kept every standard error with no warning: on 9 of 20 seededa + exp(b)fits, 20 of 20 with(1 | g)and 16 of 18 c0k fits. The check now measures the curvature again on the ridge’s floor, takes out the part that comes from the step leaving a curved ridge, and removes the direction when what is left is flat. Only a direction that involves a parameter outside a family’s primary linear predictors is asked, since a generalized linear model has no curved ridge in its coefficients.
New features
A group-level term can vary a monotonic coefficient, as in brms:
y ~ mo(x) + (mo(x) | g)addsr_g[, mox]tobsp_moxand multiplies the sum by the term’s transform, on its simplex. The population-level column is required; brms 2.23.0 drops(mo(x) | g)without one silently, and frmtmb refuses it in brms’s words. It is available in(... | g)and(... || g)terms, with predictions,ranef(),VarCorr(),conditional_effects()and frmtmb.sample’s draws. The joint log density agrees with brms’s compiled program’s at frmtmb’s estimates and conditional modes. A||term withmo()is onediag()block over the whole term, coded as brms codes it:(mo(x) * f || g)hasIntercept,fb,fc,mox,mox:fbandmox:fc. A||term withoutmo()keeps lme4’s expansion, so(f || g)still has a column for every level off.set_prior("dirichlet(alpha)", class = "simo", coef = "mox1")puts brms’s Dirichlet on onemo()simplex, and class"sbhaz"on acox()baseline;prior_dirichlet()is the object. brms’s rules come with it: a coef per simplex, the length ofalphachecked, no bounds, and onlydirichlet()on these classes.get_prior()lists asimorow per simplex and asbhazrow percox()response, as brms does. On a MAP fit the density is that of the simplex’s coordinates, with the chart’s Jacobian: the mode underdirichlet(a)is the mode ofdirichlet(a + 1/2)on the weights, pulled further toward the uniform simplex, sodirichlet(1)is not the absence of a prior (?frm, “Monotonic effects”). A concentration below 1/2 has no maximum there and is refused.
Bug fixes
A random-effect block with a scale per coefficient no longer has a
+InforNaNlog density where a log sd underflows:usandus_twith two or more coefficients,cs,toep,ar1,hetar1, andgr(cov =)andgr(prec =)with correlated slopes. Their standard deviations are floored at 1e-150 (sd_floored()), the square root of the variance floor of 0.69.0, which is below half an ulp of any sd of 1.1e-134 or more: above that the objective and its gradient are unchanged bit for bit. Over every test of the eight packages, one optimizer stepped a floored log sd below that (to -4318): a separated bernoulli fit with(1 + x | g)underfrmtmb_control(autoscale = TRUE), whose pre-fit stopped on “NA/NaN gradient evaluation” and now returns a fit with the pre-fit’s and the separation’s warnings. A sampler’s proposal below the floor meets a finite density or-Inf, so a chain that proposed there takes another path.parnames()warns once whichever generic dispatched it. The method decided from what theparnamesbinding resolved to, so frmtmb’s own generic, captured before brms loaded and called after, gave no warning.A fit could stop well short of its maximum at optimizer code 0 and without a warning when its design is badly conditioned: on raw polynomials of degree 6 with x on [1, 2], 3 of 20 fits ended up to 0.022 log-likelihood units below
lm()’s maximum, and with the response 16 times larger 19 of 20, by up to 1.9. With(1 | g), at degree 6, 5 of 20 stopped up to 0.023 short, 4 of them in silence. Such a fit now takes Newton steps from where the optimizer stopped: on the exact Hessian without random effects, and with them, on a near-collinear fixed design only, on the Hessian of the Laplace objective by differences of its gradient. A step is taken while the Hessian is positive definite and the step predicts a gain above 1e-6, kept only if it gains at least a tenth of that, and the steps are undone if they end where the Hessian is not positive definite. None of these fits stops short now, with or without the group term. A fit already at its maximum is not changed, and neither is a fit whoseoptCtrlsets one of the optimizer’s tolerances or whose optimizer is a function.A restart that raised (“NA/NaN gradient evaluation”) ended the whole fit although the run before it had a finite optimum. The fallback restarted from the lowest objective the tape had recorded, and a REML likelihood without a bound records that where the objective, solved from another inner start, is NaN (fuzz seed 20379118 under OpenBLAS’s LAPACK). That point is no longer taken, a restart that raises keeps the run before it, and the objective is left at that run’s optimum.
A restart that ended worse than the run it started from left the objective at its own last point, where
ranef()andsdreport()read it. Every run the optimizer discards now leaves the objective at the kept optimum, and so does themo()search when it refits nothing.A
mo()fit could stop below its maximum in the other sign’s cone without a warning: the search for that cone started from one vertex, at the coefficient’s own value, which is near 0 for a fit in the wrong cone, and it was skipped for an interior simplex whose coefficient is near 0. It now starts at least one of the coefficient’s units from 0, moves to the next vertex while the coefficient ends on its bound, and searches both cones when the coefficient is within 1e-3 of its unit of 0. On brms_monotonic’sls ~ mo(income) * age, 199 of 200 fits reached the exact maximum, and 194 of a further 200 (up to 0.45 short); now 400 of 400, at 1.2 to 1.4 times the objective evaluations.A group standard deviation that the likelihood does not read is “flat”, not “at zero”. The boundary message said that the sd of a group effect on a nonlinear parameter that enters no row was at zero, while it was 0.22. A boundary verdict now needs the likelihood to change when the sd moves up from its estimate, as far as 10 units on the log scale, so an sd that the likelihood reads only far above its estimate is still at its boundary.
Standard errors of a raw polynomial that the Hessian cannot resolve come from the likelihood itself. On a degree-7 raw polynomial with
(1 | g)the check named three coefficients and kept the other five with standard errors 8 to 18 times smaller thanlmer()’s, built without the two smallest directions of the Hessian, whose finite-difference noise (3e-12) is above one of them (8.9e-15 of the largest). A direction that the Hessian has no digits for is now measured again, with the confirmed small directions next to it, by second differences of the objective; all eight standard errors are now within 0.5 percent of the exact ones from a QR of the whitened design (lmer()’s own are 9 percent below them). A direction the objective cannot confirm is still flat.An ill-conditioned design is named as one. On a degree-8 raw polynomial the smallest curvature is below machine epsilon of the largest, and 0.69.0 said that the data do not determine six of the nine coefficients and kept three. One double-precision covariance cannot hold that curvature beside the others, so the coefficients it carries still have no standard error, and neither does a prediction that reads them. The warning now says that the design is ill-conditioned and that centering the covariate or
poly()removes this, and, when it applies, that the optimizer stopped short of the maximum, with the log-likelihood a Newton step would gain (up to 4.5 on these fits). Terms beside the polynomial that do not read those coefficients (a covariate, a factor, sigma) keep their standard errors.
Extension API
-
The sampling API exports
simplex_frame_terms()(?frmtmb-sampling-api): every simplex of a fit’s frame in the shapemo_frame_terms()gives amo()term, themo()simplexes and then eachcox()baseline under brms’s namesbhaz[k].mo_chart_frame()now sets acox()baseline’s chart too. frmtmb.sample 0.18.0 samples every simplex through them.
frmtmb 0.69.1
The Ubuntu and macOS checks of 0.69.0 failed where the Windows ones passed. dev/ci691-findings.md has the causes and the numbers.
Bug fixes
An nlminb fit reports the objective at its own estimates. When PORT stopped on a finite trial point it had rejected,
fit$opt$parwas that trial andfit$opt$objectivethe best value, at another point. 0.69.0 repaired this only when the trial’s objective was not finite. On acs()ordinal mixture the two values differed by 1 to 9 ulps in 10 of 100 runs; the estimates now come from the best point evaluated, as they already did after a non-finite trial.Three tests asked for an outcome that rounding decides: the standard errors of an
xbeta()fit whose Hessian sits at the standard-error check’s flat threshold, the convergence code of a degenerate three-component ordinal mixture, and a list of lost standard errors that can be empty. They now ask for the properties that hold on every platform.
frmtmb 0.69.0
Four lanes, each with an adversarial review and punch rounds: ciharden (tests and CI made independent of platform rounding, CI pinned to ubuntu-24.04 with a Stan job on Ubuntu 26.04 beside it, less memory in the gp() kriging covariance), surface (the brms post-processing calls a ported script meets on a maximum-likelihood fit now answer or refuse by name), setier (which standard errors a fit can report, a boundary fit said as lme4 says it, separation named) and optima (fits that stopped short of the maximum likelihood: mo(), the probit tails, cs() ordinal mixtures, nlminb’s rejected trial point). Each lane’s dev/<lane>-findings.md has the validation, the numbers and the scripts, and dev/reviews/2026-10-07-<lane>.md has the review; dev/round-20261007.md is the round’s record. In brms’s own ported suite, bin 1 now passes 390 of 494 assertions, up from 389 at 0.68.1.
Breaking changes
stancode(),standata()andpp_mixture()are frmtmb’s generics, with methods for a fit; they were frmtmb.sample’s, with methods for draws only.stancode(fit)andstandata(fit)stop with the reason and namebuild_objective()andfit$frame. Before, they said “no applicable method”, and with brms loaded they reached brms’s default method, which said “Data must be specified using the ‘data’ argument” about a fit that has data. frmtmb.sample re-exports the three generics and needs this version.The pooled table of
frm_multiple()names its rows as brms does:Intercept, not(Intercept), asfixef()of each fit names them. A distributional parameter without a formula, pooled on its link scale, issigma_Intercept.update()drops a stored prior the updated model cannot take, as brms’supdate()does, and says which in a message, where brms is silent. Updatingbrms_overview’sfit2, which removes the last correlation, stopped at the class-widelkj(2)prior; it now fits without it. A prior passed toupdate()itself replaces the stored one, where brms merges the two, and is checked asfrm()checks it.A standard error built on a row of noise is gone, on every platform. The standard-error check kept sdreport()’s inverse of the outer Hessian whenever it gave positive variances, even when a parameter’s row of the finite-difference Hessian was pure noise. A variance component at zero then got a standard error of about 10^3 to 10^5 on its log scale, or
NaNand a warning, by the sign of that noise:(1 | Subject/a)onsleepstudygave 375,150 with the reference BLAS andNaNwith OpenBLAS. The check now keeps an inverse only where its third tier would remove nothing: no row within the Hessian’s noise, no log standard deviation at its edge (below), no parameter held by a bound, and no direction flatter than 1e-9 of the largest unless the likelihood curves along it. Such a parameter now has aNaNstandard error invcov(),summary()andconfint(), and the other standard errors are those of the Hessian without it. A fit that loses nothing keeps sdreport()’s covariance bit for bit. A parameter held by a bound is now reported on every fit, where before it was only when some unrelated direction was nearly flat.A covariance parameter at the edge of its parameter space is a boundary fit, said with a message. A group standard deviation is at its edge when moving it a factor of 7 toward zero changes the log-likelihood by at most 1e-6; a correlation or mixing parameter when moving it further toward its limit does the same; the one correlation of a block whose standard deviation is at zero is undefined. Its lost standard error is reported as lme4 reports a singular fit: one message, “Boundary (singular) fit: …”, of class
frmtmb_boundary_fit, naming the parameters with brms’s names (sd_g__Intercept), and not the standard-error warning. An sd near zero is first moved up to 0.01, 0.03, 0.1, 0.3 and 1 times the residual sd (1 on the link scale without one): if any gains, the fit stopped short of its maximum, and the warning says so (“the fit stopped short of the maximum”; at a non-zero code the convergence warning stays). Of 24 random-intercept fits that stop 1e-4 to 0.825 below lme4’s log-likelihood, none gets the boundary message. Underquadrature = TRUEno boundary verdict is given, since that check cannot run on a quadrature objective: such an sd keeps the “flat” standard-error warning, as before. On 600 simulated random-intercept, random-slope and binomial fits the message fired on 267 of the 269 that lme4 calls singular and on none of the other 331; on larger single-sd designs (60 to 300 groups) on 16 of the 17 lme4 calls singular and none of the other 103; on random slopes with a correlation of 0.95 or 0.99, on 19 of the 33 lme4 calls singular (20 with OpenBLAS) and on 2 others, whose correlation lme4 puts at 1 - 2e-7 and 1 - 4.5e-6. When nlminb stops with “singular convergence (7)” there, the gradient is withingrad_toland every lost parameter is at its edge, the fit gets this message and not “Optimizer did not report convergence”, as lme4’s checkConv() does (67 of 69 such random-slope fits).frmtmb_control(check_se = "ignore")silences it, and"stop"makes it an error atfrm(), which now builds the check there even when it would otherwise wait for the firstsummary(). A smooth,gp()orhsgp()term at its own limit (a smooth reduced to its unpenalized part, as mgcv reports in silence) loses those standard errors with no message: 91 of 140 gamSim fits do.summary()lists both under “Parameters without a standard error”, anddiagnose()lists the standard deviations as singular.VarCorr(),confint_varcorr(),predict()andfitted()do not warn again about them;predict()’s draws hold them at their estimates.A small eigenvalue of the outer Hessian keeps its standard error only if the likelihood is quadratic along it. The check takes the symmetric second difference
f(p + t) + f(p - t) - 2 f(p)along the direction, at the step the curvature says loses 2 *grad_toleach way and at half of it; it keeps the direction when the full one is at least 2 *grad_toland 3 to 5.5 times the half one (a quadratic gives 4). Symmetric, so that a fit not exactly stationary along the direction keeps it: an identified poisson or bernoulli pair at a correlation of 1 - 5e-11 and raw polynomials of degree 5 or 6 on[1, 2], alone or with a random intercept, keep every standard error, close to glm()’s, lm()’s and lme4’s. A straight step off a curved ridge, which loses as the fourth power, does not keep it.A nonlinear ridge on a fit with random effects loses its standard errors.
y ~ a + bwitha ~ 0 + fandb ~ 1 + (1 | g), where onlya_k + bis identified, kept finite standard errors from the finite-difference Hessian, and a prediction ofareported 822,571. The ridge’s coefficients now haveNaNstandard errors, and a prediction along the ridge getsNaNand one warning, as without random effects. The identification warning for nonlinear parameters now names only the coefficients the standard-error check lost: at a correlation of 1 - 1e-9 between two covariates it named coefficients whose standard error, 1268, lm() and the exact Hessian both give.Separation is a warning of its own, named once the fit has run far enough, whatever code the optimizer returned. In a
bernoulli(),binomial()orbeta_binomial()mean, “The data separate the outcomes (complete separation)” or “(quasi-complete separation)”, with the coefficients involved (the first five and the count), classfrmtmb_separation. The fit is checked against the definition, along the direction it ran off in, so a large but finite effect is never named (0 of 338 non-separated fits, 162 of 162 separated ones, logit link; probit and cloglog are named too). It replaces the convergence warning it explains, and the coefficients it names show no standard error (NaN) on every path. A fit stopped very early may not have run far enough to show it: complete separation ateval.max5, 10, 20 and 40 was named on 5, 16, 19 and 20 of 20 fits, a level of a factor with no successes at 5, 10 and 20 on 0, 0 and 20; the misses keep the convergence warning. A factor too wide to decompose (above 2000 columns) has each column tried on its own. Before, onyb ~ z + xwithyb = z > 0, nlminb stopped at code 9 with OpenBLAS and the user read only “Optimizer did not report convergence”; on a level of a factor with no successes, nothing was said. The check keeps a sparse design sparse: 0.4 s on a 1e6-row fit with a 500-level factor that took 719 s.A
mo()fit reaches the maximum likelihood. The simplex of a monotonic term was held as softmax coordinates, which put a step of 0 at an infinite coordinate. A fit whose step ran toward 0 stopped on that plateau with code 0, wherever the likelihood still rose, or at the local maximum of the coefficient’s other sign. On brms_monotonic’s ownls ~ mo(income) * age, 84 of 200 data sets ended more than 1e-6 below the exact maximum, 53 by more than 0.01 and up to 1.95, and 18 had standard errors that were not finite. Now 199 of 200 reach it to 1e-6 (the other is 0.049 below), every standard error is finite, and no fit warns. The simplex is held as the coordinates of stereographic projection onto the unit sphere, whose squared components are the simplex:zeta = 0is still the uniform simplex, and every face is at a finite coordinate, where a step of 0 keeps a finite curvature. After the fit, a term whose simplex is on a face is refitted with its coefficient held at the other sign, and the better fit is kept;fit$opt$mo_searchrecords the refits andfrmtmb_control(mo_search = FALSE)turns them off. The search costs objective evaluations, and the factor grows with the model, since each refit is of the whole model: 2.4 times 0.68.1’s over the 200 interaction fits, 1.3 times on the plainls ~ mo(income)(which reached the maximum before and does now), and 3.3 to 3.8 times on a model of 5000 rows with a random intercept over 100 groups. Thezeta<j>values thatconfint(),vcov()andstartread are these coordinates, which have no reading of their own;summary()now reports the simplex itself, as brms does, under “Monotonic Simplex Parameters” (moincome1[1], …), with delta-method standard errors; a weight at 0 or 1 has none there (the chart folds at the face, and the delta method would give an interval of [0, 0] or about [1, 1]), and a line under the table says so.dev/optima-findings.mdhas the measurements.The probit, cloglog and softit links keep their log-odds in the far tails. The probit’s was
log(pnorm(eta)) - log(pnorm(-eta)), infinite past|eta| = 38.2and with a NaN derivative on 24 of 2001 points inside it; an ordinal probit’s density was then NaN. It now reads both logs frompnorm(log.p = TRUE): within 7e-16 of Rmpfr in value on[-200, 200]and 3.6e-12 in derivative. The old form erred by up to 4.3e-12 where it was finite, and probit fits move with it: estimates by up to 9e-10 relative, log-likelihoods by up to 3.4e-13. Past|eta| = 1000, where RTMB’s derivative ofpnorm(log.p = TRUE)degrades, it continues as the tail’s leading term. The cloglog’s and softit’s log-odds lost their derivative beloweta = -708and their value below -745; below -40 they areetaitself, which they equal there to double precision. Above -40 both are unchanged in value, and their fits move by an ulp of the derivative: estimates by up to 3.6e-12 relative.
New features
pp_mixture()on a mixture fit, ordinal mixtures included: the component probabilities at the estimates in brms’s observations-by-statistics-by-components array, with components namedP(K = k | Y). The quantile columns are a Wald interval on the logit, andEst.Erroris the standard deviation of that same logit-normal law, integrated numerically: the delta-method error of the probability itself understates the spread near 0 and 1 by orders of magnitude. On a 300-row two-gaussian mixture against brms 2.23.0’spp_mixture()(4 chains of 1500 draws),Est.Errorwas a median 1.11 of brms’s posterior SD over all rows (the delta method’s was 0.36), 0.84 to 1.27 by bin for probabilities between 0.001 and 0.999, and up to 4.8 times it within 0.001 of an edge; the interval was a median 1.04 of brms’s width, and every estimate was inside brms’s 95% interval. brms’s draws arguments are refused by name.fixef(),summary()andconditional_effects()of afrm_multiple()result pool the imputations by Rubin’s rules, where brms’sbrm_multiple()combines their draws.fixef()has brms’s rows, ordinal thresholds included, and a t interval on the Barnard-Rubin degrees of freedom. Its estimates agree withmice::pool()oflm()onnhanesto 4.1e-6; its standard errors are 0.93 of mice’s, because each imputation’s variance is the maximum-likelihood one, and its complete-data degrees of freedom aredf.residual()of the fit, which countssigma(20 there, wherelm()has 21), asanova()of afrm_multiple()result already takes them.summary()prints brms’s blocks withdfandfmicolumns.conditional_effects()pools every grid point of the first imputation’s grid on the scale its Wald band is symmetric on;method = "predict"and the bootstrap and profile bands are refused by name.add_criterion()exists, with brms’s criterion names. On a fit it refuses each criterion by name: every one is a posterior quantity, andAIC()andBIC()are the maximum-likelihood comparison. On draws, frmtmb.sample’s method stores it as brms does.
Bug fixes
plot()of a fit refuses brms’splot.brmsfit()arguments by name (N,nvariables,variable,regex,pars,fixed,combo,bins,theme,plot,newpage), with the reason andplot(frm_sample(fit), ...)for brms’s display. It suggestedxforN.summary(fit, waic = TRUE)is refused by name with the reason. brms 2.23.0 ignores the argument.fitted()on an ordinal fit whosediscpredictor has no fixed column (disc ~ 0 + gp(x, k = 6),disc ~ 0 + (1 | g)) stopped with “requires numeric/complex matrix/vector arguments”, in sample and on new data.frm_sample(fit)on an exacty ~ gp(x)fit moves. A one-scale dense block (gp(),ou(),homcs(),homtoep(), the spatial structures andgr(cov =)with one coefficient) whose log sd passed about -372 had the zero matrix as its covariance, and its field’s log density was+Inf. The sampler’s first long step from the mode reached such a point and stayed there: acceptance 0, step size NaN, every transition divergent. The variance is now floored at 1e-300, which is the variance itself, bit for bit, above 9e-285, so no fit changes.(cs(1) | g)with brms attached after frmtmb got the internal “variable lengths differ (found for ‘.frm_cs(1)’)”; it gets the refusal it gets without brms. The refusal ofconditional_effects(method = "predict")oncategorical()names that family, where it said “ordinal family”.ranef(condVar = TRUE)on a fit that lost a standard error reads the repaired joint covariance that predictions read. It read sdreport()’s own conditional variances, built on the indefinite inverse: on agr(g, by = f)fit they ran from 0 to 99 times the repaired ones.frm()read its estimates, and started its restart, from the trial point nlminb had rejected when nlminb stopped on one (“false convergence (8)”), whilelogLik()reported the best point. The best point the optimizer evaluated is now the fit’s.A fit whose family declares a stationary point (
skew_normal()) and that has random effects reported the random effects’ modes of the last escape it tried, not of the optimum it kept. Onskew_normal()with(1 | g), seeds 1 to 40, the escape ran on 14 fits and on 5ranef()was off by 0.002 to 0.143.cs()on a cumulative component of an ordinal mixture no longer stops with “NA/NaN gradient evaluation”.mixture(cumulative(), sratio())withcs(z)died on 7 of 20 data sets andmixture(cumulative(), cumulative())on 4 of 20; both now finish on all 20. Two causes: the rejected trial point above, and two thresholds of one row that the line search had brought to the same double, where the component’s density was-Inf, the mixture’s finite and its gradient NaN. That touch is now as undefined as the crossing past it. Most of these fits still report non-convergence, correctly: their optimum is where a row’s category closes in one component, and the gradient there does not vanish. On the 18 of 20mixture(cumulative(), sratio())fits that stop with “false convergence (8)”, some row’s two thresholds are within 2.3e-13 of each other.
Performance
- The kriging covariance of a
gp()term no longer holds extran x ncopies of the prediction grid:gp(x, by = <numeric>)held two more, and every exactgp()one more for writing its diagonal. It is the same matrix to the bit.
Extension API
-
The sampling API exports
mo_simplex(),mo_coords(),mo_chart_frame(),mo_frame_terms()andsummary_mo_frame()(?frmtmb-sampling-api), so that an extension can read amo()simplex in the coordinates the fit holds it in, or in the softmax chart a sampler needs, and give the weights brms’ssimo_names. frmtmb.sample 0.17.0 samples amo()simplex through them.
frmtmb 0.68.1
The Ubuntu checks of 0.68.0 failed where the Windows and macOS ones passed. dev/cifix-findings.md has the causes, the numbers and an OpenBLAS build of R for Windows that reproduces them.
Breaking changes
-
A fit with random effects and a parameter without a standard error gives its predictions a covariance. When the outer Hessian lost a standard error (a smoothing or group sd at zero, for example), the joint precision built on it was not positive definite, and inverting it gave grid rows of
s(x, by = fac) + gp(x)coefficient variances as low as -0.10 (with OpenBLAS; -0.016 with the reference BLAS).frm_linpred(se.fit = TRUE)then reported the kriging variance alone on those rows, 0.00076 where the standard error is at least 0.064, and too small a variance on others. The joint covariance now propagates the outer parameters’ uncertainty only along the directions the fit determines, as the fit without random effects already did. It is conditional on the lost directions: when each lost parameter is a direction of its own, that is the covariance with them held at their estimates.frm_linpred(),frm_lp_basis(),frm_joint_cov()and everything built on them change on such fits, and only on them.frm_joint_cov()shows a lost parameter’s row and column asNaN, asvcov()does, and returns their rows aslost_pos.vcov()under REML, which reads the joint precision directly, is not changed.
Bug fixes
On a fit with random effects, a prediction that moves along a direction the fit does not determine gets a
NaNstandard error and one warning, as it already did without random effects. Ony ~ x + (1 | id)with one observation per id, where sigma and the id sd lose their standard errors,frm_linpred(dpar = "sigma", se.fit = TRUE)andfitted(dpar = "sigma")reported a standard error of 0 with no warning, under ML and REML.Rows of one prediction at the same unseen exact
gp()position get bit-identical kriging weights, variance and covariance on every BLAS. Each distinct position is kriged once. With OpenBLAS the rows differed by a few ulps, so a difference curve across a factor at one position kept 9 ulps of the residual it cancels.Tests that passed on Windows by a platform accident now pass on Ubuntu too: the separation fit of
test-se-check.Rhas a budget at which nlminb converges on both, and two fits whose sd runs to zero allow the standard-error warning that OpenBLAS’s rounding gives them.
frmtmb 0.68.0
Five lanes, each with an adversarial review and punch rounds: fixes (emmeans() on transformed predictors, trunc() and se() in conditional_effects(), the ordinal leftovers of 0.67.0, brms’s s() null-space basis, the nonlinear flat-direction warning), gpby (gp(x, by = ) with brms’s arguments and prior classes, and the exact gp() kriging covariance at new positions), ordmix (ordinal mixtures, hurdle_cumulative() with thres(gr = ) and cs()), nanse (a fit says which parameters have no standard error, and why, and frm_allfit() starts where lme4::allFit() does) and vigport (the brms vignette port measured again, records only). The consolidation lifted the refusal of cs() on cumulative() and made conditional_effects(method = "predict") report the median, both the user’s decisions of 2026-10-06. Each lane’s dev/<lane>-findings.md has the validation, the numbers and the scripts, and dev/reviews/2026-10-05-<lane>.md (2026-10-06-nanse.md for lane nanse) has the review; dev/round-20261005.md is the round’s record. In brms’s own ported suite, bin 1 now passes 389 of 494 assertions, up from 387 at 0.67.0.
Breaking changes
frm()builds the Hessian at the optimum and warns when a parameter has no standard error. The warning starts “Standard errors are not available for k of n parameters”, names each parameter, and gives the reason: a bound holds it, the data separate the outcomes, the likelihood is flat along it, a step along it raises the log-likelihood (confirmed with two objective evaluations, else it is called flat), or its Hessian row is not finite. Before, a converged fit could returnNaNfor every standard error with no warning:ls ~ mo(income) * ageon the brms_monotonic vignette’s own data code did so on 55 of 200 data sets at optimizer code 0. Now 26 of those 200 fits get this warning, 17 get a convergence warning, none loses a standard error silently, and the warning fires on none of the 357mo()fits whose standard errors are all finite. In the eight test suites it fires 43 times outside its own tests, every time at optimizer code 0 on a fit where sdreport() gave a non-finite standard error. A fit that did not converge gets its convergence warning instead. An earlier warning explains only the parameters it names: a grouping factor with one level its variance, an observation-level effect its variance andsigma, a family’s own fit-end check (a threshold no row places, an unidentifieddiscintercept,zero_one_inflated_beta()’scoi) all but bound-held parameters. A check set to"ignore"explains nothing. The cost grows with the number of parameters. Without random effects the check reads the exact Hessian, about one gradient per coefficient: 8 to 23 percent of the fit on seven small models and 19 to 24 percent on a glm with 301 coefficients. With random effects it builds the finite-difference Hessian, two gradients per outer parameter, when the model has at most 10 outer parameters or those gradients are at most a quarter of the evaluations the optimizer made. Otherwise the check waits for the first use of a standard error and warns there, also when that use isfixef(). The rule counts work and does not read the clock, so the same call warns in the same place on an idle machine and a loaded one (dev/nanse-rev2-determinism.R: 40 of 40 at fit time in both). Mixed models with 14 to 23 outer parameters were checked at fit time for 19 to 23 percent; with 42 to 403 the check waited, and the fit cost 3 percent or less.summary()reuses the finite-difference Hessian.frmtmb_control(check_se = "ignore")skips the check and"stop"makes it an error (dev/nanse-findings.md).A parameter without a standard error no longer takes the others with it. Where sdreport()‘s inverse fails, the covariance is recomputed. First the Hessian is scaled to unit diagonal, and on a model without random effects the exact Hessian replaces the finite-difference one. Of the 55
mo()fits above, 47 get every standard error back, equal to the same Hessian inverted with the saturated simplex coordinate held to 2.9e-5 relative, and 8 lose only one to three parameters. Ana * exp(b * x)fit withxspanning 0 to 1e5 gets its standard errors back too; the finite-difference step of 1e-3 inb = -2e-5had given a diagonal of 1.3e96. Where both repairs fail, a parameter held by a bound (test-backlog “The covariance machinery is not bound-aware”) and a simplex weight whose Hessian row is exactly zero lose their standard errors, and so does every parameter with a measurable part along a direction the likelihood does not determine, however many share it:y ~ a + bwitha ~ 0 + fat 60 levels loses all 60acoefficients andb. The other parameters keep theirs.summary()lists the lost parameters with the reason instead of a bareNaN. A prediction, emmean, hypothesis orVarCorr()entry that moves along a removed direction gets aNaNstandard error, andfitted(),predict(),frm_linpred(),conditional_effects(),emmeans(),hypothesis()andVarCorr()warn once per call with the number of such rows; one that does not keeps its standard error. Soconditional_effects()keeps its bands on themo()fits, and ony ~ x + (1 | g1) + (1 + x | g2)with theg2block at variance 0,VarCorr()keepsg1’s sd error (0.233, ashypothesis()gives it) and the residual sd’s (0.0794). A parameter that keeps its standard error does not pass a removed direction to the predictions through its own small part in it, and aconditional_effects()plot whose band isNaNdraws the curve alone instead of stopping on “need finite ‘ylim’ values”. Under REML orprofile = TRUEthe coefficients’ standard errors come from the joint precision, which this does not repair, and the warning says so. A fit that did not converge keeps sdreport()’s own covariance.frm_allfit()starts every optimizer where lme4’sallFit()does. The refits started from zero, so on a nonlinear model 3 of 4 refits failed andbobyqareported success 155 log-likelihood units below the fit it was checking. They now start at the original fit’s estimates (start_from_mle = TRUE, lme4’s default) or, withstart_from_mle = FALSE, at the original fit’s own start, including itsstart =. The printed table compares every refit with the best log-likelihood reached and marks a success code more thangrad_tolbelow it as “converged elsewhere”. A failed refit keeps its error message in$errors. The fit object stores its caller’sstartasfit$start.A smooth’s null-space coefficient is brms’s.
s()is built withmgcv::smoothCon(diagonal.penalty = TRUE), as brms builds itsXsandZs, sofixef()’ssx_1(andsigma_sx0_1) is brms’s coefficient and a prior withcoef = "sx_1"or a hypothesis on it is about brms’s parameter. Before, it was brms’s coefficient times a data-dependent factor (5.0 to 7.7 on gamSim data, of either sign). The design is now brms’s standataXsandZsbitwise for every single-penalty smooth,by =and 2-Ds()included;t2()is unchanged. The model is a reparameterization of the old one, but the optimizer’s path changes, so a different few fits stop at a worse optimum. On 1020 gamSim fits per build (gamSim seeds 1 to 60, 18 smooth shapes; the review’sdev/fixes-rev2-smooth.Rand-smooth-cross.R): convergence codes other than 0 in 11 against 16 on the old basis, non-finite fixed-effect standard errors in 37 against 31, and 9 fits on each build more than 1e-3 below the bestlogLikeither build or mgcv found, 6 of them the same fits; each fit only one build got wrong reaches the other’s optimum from the other’s smoothing SDs. The optimizer steps the null-space coefficients in unit-SD coordinates, without which a smoothing SD at zero ran far down its log scale and left NaN standard errors far more often.A nonlinear fit warns when its nonlinear parameters’ coefficients are not identified at the optimum, naming the coefficients: when the likelihood is flat there along a combination of them, as with two parameters that enter only through their sum or difference while their formulas share a term (
a + bwitha ~ 1 + xandb ~ 1 + x), or a product of two intercepts. Such a fit used to return one point of the ridge without a word. The check reads the Hessian of those coefficients at the fitted point, which costs 2 k gradients for k coefficients (0.61 s of an 11.3 s fit at n = 200000). A curved ridge whose curvature does not vanish to 1e-9 there (exp(a)^k) is not flagged;vcov()andsummary()still report its standard errors as not finite. A prior that reaches every flat direction removes the warning, and then the prior alone sets the split, so a weak one gives very wide intervals.conditional_effects()holds the variable of atrunc()bound or anse()term at its mean whenconditionsdoes not set it, as brms does, where it refused the display. An expression bound is evaluated on the grid, where the response is at its mean too, sotrunc(lb = min(y) - 1)ismean(y) - 1, as in brms. The expected value is the truncated mean at the held bound and agrees with brms 2.23.0.confint()andvcov(full = TRUE)name an ordinal fit’s internal threshold parameters by what they are, where they readtau_raw_<k>:Intercept[1],Intercept[a,2]ordeltawhere the internal parameter is that one, andlog(Intercept[2] - Intercept[1])orlog(delta)where it is a transform. Acs()coefficient takes itsfixef()name.parm =takes these names, brms’sb_Intercept[1], and the old template names.gp()defaults toiso = TRUE, as brms 2.23.0 does. A multi-dimensionalgp(x1, x2)now shares one lengthscale over the Euclidean distance;gp(x1, x2, iso = FALSE)gives one per dimension, which was the old default. The code comment that called the old default brms’s was wrong: brms’sgp()signature hasiso = TRUE, and its Stan program forgp(x, z)declares one lengthscale (dev/gpby-brms-explore.R).Prior class
"sd"no longer reaches agp()term. brms priors a GP’s marginal standard deviation as class"sdgp"and its lengthscales as class"lscale", and frmtmb now has both classes (see New features).set_prior(class = "sd")on a model whose only random-effect block is agp()is refused, and the message names"sdgp". A brms prior table withsdgpandlscalerows, which was refused by name, now carries over row for row.get_prior()lists no class"sd"rows for agp()term. It listedsdrows for the GP’s block, which no prior could reach as brms names it; it lists thesdgpandlscalerows brms’sdefault_prior()lists instead.summary()$gplabels a lengthscale row with brms’s name,lscale(gpxzx)andlscale(gpxzz)where it wrotelscale(gpxz[1])andlscale(gpxz[2])forgp(x, z, iso = FALSE)(dev/gpby-p1-sumgp.R). Code that indexed the table by the old row names must use the new ones.cs()in the formula of a parameter other than the latent predictor is refused, in brms’s words (“Category specific effects are only supported for the main parameter ‘mu’”). The offsets of every predictor went into the one slot the ordinal densities read, sobf(y ~ x, disc ~ cs(z))fittedy ~ x + cs(z)under disc’s name, its log-likelihood that model’s to 1.1e-9, with the coefficients reported asdisc_z[k](dev/ordmix-base-behavior.R).conditional_effects(method = "predict")reports the median of its simulated responses asestimate__, as brms reports the median of its predictive draws. It reported the expected response, whichmethod = "epred"still shows. The band is unchanged, and the estimate is the 50% point of the same draws: a whole count for a discrete family whenndrawsis odd, inside the bounds of a truncated one. At a Poisson fit’s x = -1, 0, 1 it is 1, 2, 4, where it was 1.46, 2.52, 4.35, and brms’s median at the same parameters is 1, 2, 4 (dev/fixes-ce-pred.R,test-ce-predict-median.R). frmtmb.sample’s draws route reported the median already. The user’s decision of 2026-10-06.
New features
default_prior()lists one class"Intercept"row per threshold,coef = "1","2", …, under flexible thresholds, and per level ofthres(gr = ), as brms does, andset_prior(class = "Intercept", coef = "2")puts a density on that threshold. It is refused underthreshold = "equidistant", as in brms.The compatibility table has rows for
discand the"equidistant"and"sum_to_zero"threshold structures.gp(x, by = f), with brms’s semantics.gp()takes brms’s whole argument list:by,k,cov,iso,gr,cmc,scaleandc(k = NAandby = NAmean none, as in brms). A factorbyfits one GP per level undercmc = TRUEand an intercept GP plus contrast GPs undercmc = FALSE, each over its own rows, scaled and centered over those rows alone and with its own standard deviation and lengthscales, asbrms:::data_gp()builds them; a numericbymultiplies one GP.gr = FALSEkeeps one latent value per row andscale = FALSEkeeps the inputs unscaled. On six Hilbert-space forms (factor and numericby,cmc = FALSE,gr = FALSE, isotropic and anisotropic two-dimensional) brms’s Stan log density at frmtmb’s estimates equals frmtmb’s joint density to 2.8e-14 and its gradient in the latent values vanishes to 2.6e-14 (dev/gpby-lpcheck.R); the exact form’s marginal likelihood is the closed form to 1e-8 relative (tests/testthat/test-gp-by.R). A by-level the fit never saw is refused with brms’s own message,allow_new_levels = TRUEincluded, as brms refuses it.cov =other than"exp_quad"is refused: the Matern and exponential kernels are not implemented.brms’s names for a GP’s parameters.
variables()andhypothesis()readsdgp_<term>andlscale_<term>(sdgp_gpx,sdgp_gpxfafor levelaofgp(x, by = f),lscale_gpxzxfor a non-isotropic term), on brms’s scales, andsummary()$gplabels its rows the same way (see Breaking changes). Before,variables()listed neither parameter, so brms’svariables(fit6)test could not transfer.Prior classes
"sdgp"and"lscale", brms’s, withcoefthe sub-GP brms names ("gpxfa"). Both are on brms’s natural scales with the log-Jacobian applied; the lengthscale is on the inputs brms divides by their largest distance.get_prior()lists the rows brms’sdefault_prior()lists.frm_lp_basis(extra_cov = TRUE)returnsextra_cov, the full covariance over the rows of the variance that is not coefficient uncertainty: an exactgp()’s kriging residual at an unseen position, which every unseen row of one field shares, and a new grouping level’s draw, which every row at that level shares. Its diagonal isextra_varexactly. It is a sparseMatrixwhere its sources cover few entries, an empty one (nothing allocated) on a fit with no such source, and dense otherwise: a 2000-row grid past an exactgp()’s data peaks at 106 MB in the call (dev/gpby-p1-mem3.R). One call on a stacked grid gives the covariance between two grids.frm_extra_cov_deriv(), the covariance of theorder-th derivative of that same part, built from its sources: an exactgp()’s kriging residual by the squared exponential kernel’s closed form derivatives, a new grouping level by differencing its draw’s design rows. Differencingextra_covitself divides its rounding by the step to the fourth power at order 2, which gave a standard error of 0 past an exactgp()’s data and up to 1.5 for the curvature of a term linear in the variable, whose true value is 0 (dev/reviews/2026-10-05-gpby.md, B1).emmeans()on an exactgp()at a position the fit did not see. Refused until now; the kriging covariance across the grid entersV, so a mean at an unseen position carries it and a contrast between two rows at one position cancels it.mixture()takes ordinal components, as brms does:cumulative(),sratio(),cratio()andacat()in any combination, each with its own link,disc<k>and threshold structure, orhurdle_cumulative()for every component.order = "none", brms’s default for them, gives each component its own thresholds,b_mu1_Intercept[k],b_mu2_Intercept[k], andorder = "mu"gives all of them one shared vector, brms’sfixed_Intercept, which a sum-to-zero component centers (with components that have amuintercept,order = "mu"stays refused).thres(x = ),thres(gr = ),cs()per component,theta<k> ~anddisc<k> ~work. The category probabilities thatfitted(),predict(),simulate(),conditional_effects()and frmtmb.sample’sposterior_epred()andlog_lik()report are the theta-weighted sums of the components’. The log density equals brms 2.23.0’s compiled program at the optimum and at three perturbed parameter vectors to at most 2.5 ulp on the 20 mixture shapes ofdev/ordmix-lpcheck.R; a probit component whose latent distance from a threshold passes about 38 has aNaNdensity where brms’s is finite (one point of the review’s three-component probit, sratio and acat mixture).fixef()has brms’s rows in brms’s order.default_prior()lists brms’s threshold rows, andset_prior(class = "Intercept", dpar = "mu1")(with nodparunderorder = "mu") lands where brms puts it. A sum-to-zero component lists no class"Intercept"rows, where brms lists one per threshold, as for one sum-to-zero family. brms’s equidistant mixture does not compile (its parameters block declares a baredeltaper component, and its body readsdelta_mu<k>); frmtmb fits the model the body describes and names itdelta_mu<k>.groups =and ahurdle_cumulative()component beside one without a hurdle are refused. A mixture whose likelihood has a flat direction by construction warns: no predictor in any parameter, or hurdle components whosehu<k>and mixing weights have none, unless priors hold enough of those parameters (a threshold prior holds none of the hurdle’s); or shared thresholds with nomu<k>predictor on components of one family, link anddisc. A fit whose component ends at a degenerate boundary warns and names it: a step function of its predictors (doubling its latent distances costs the log-likelihood less than 0.1), a threshold no row is near, or two thresholds that are the same number. Maximum likelihood for an ordinal mixture often has its supremum there (22 of 160 simulated two-component fits; the warning fires on 20 of them and on none of the other 138, nor on any of 167 sound fits with strong predictors under four links), so compare several starts, or hold the components with priors.hurdle_cumulative()takesthres(gr = )andcs(), as brms fits them. Each group’s density keeps the hurdle, the counts are over the categories above it, andcs()comes off each row’s thresholds, with brms’s warning that the effects are experimental. The log density equals brms 2.23.0’s compiled program to at most 1.6 ulp under the logit and the probit, withhu ~,disc ~and every threshold structure (dev/ordmix-lpcheck.R). A row whosecs()offsets cross two thresholds has a negative category probability: its density andfitted()areNaNthere (brms returns the negative difference), andsimulate()givesNA.cs()withthres(gr = )stays refused, as in brms.cs()oncumulative(), as brms 2.23.0 fits it, with brms’s warning that category-specific effects for the family are experimental. It was refused. Rowireads the thresholdstau_k - cs_ik, brms’sIntercept - transpose(mucs[n]), and the log density equals brms’s compiled program to at most 0.9 ulp at the optimum and at three perturbed points on six shapes: the logit and the probit, a predictor besidecs(), twocs()terms, a factor incs(),disc ~ 0 + z, equidistant thresholds and a mixture withsratio()(dev/rel068-cs-lpcheck.R). The offsets can make a row’s thresholds cross: such a row’s density andfitted()probabilities areNaN, as brms’s density is (brms’sposterior_epred()returns the negative difference), andsimulate()givesNAthere.predict()givesNAproportions for a row that crosses at the estimates; a row that crosses only in some of its simulated parameter draws reports the proportions over the other draws, and the call warns with the draws dropped per row, here and onhurdle_cumulative()(the user’s decision of 2026-10-06).hurdle_cumulative()and acumulative()component of an ordinal mixture takecs()the same way and warn as brms does, once per component. A family that takes nocs()is refused in brms’s words, “Category specific effects are not supported for this family”. The user’s decision of 2026-10-06.Ordinal mixtures and the per-threshold features meet.
default_prior()lists the per-threshold class"Intercept"rows of each component underdpar = "mu<k>", and of the vector anorder = "mu"mixture shares once with nodpar, as brms lists them;set_prior(class = "Intercept", coef = "2", dpar = "mu1")puts a density on that threshold alone.confint()andvcov(full = TRUE)name a mixture’s thresholds by what they are,mu1_Intercept[1]andlog(mu1_Intercept[2] - mu1_Intercept[1]), and the shared vector once, asIntercept[1]. The compatibility table’s cells fordisc,"equidistant"and"sum_to_zero"withmixturesay what the release does: equidistant is refused underorder = "mu", as in brms.
Performance
-
The kriging draw on a high-rank grid is faster. Past a quarter of the unseen positions,
gp_krig_factor()takes the whole conditional covariance and LAPACK’s pivoted Cholesky instead of its column loop: 0.020 s against 0.071 s at rank 400 of 400 positions and 0.161 s against 0.293 s at rank 499 of 1000, with the law unchanged (dev/rel068-krig-rank.R). The low-rank regime the loop was built for is untouched.
Bug fixes
A nonlinear fit whose default start a prior placed outside the family’s domain stopped at that start. A prior’s location places every coefficient of a nonlinear parameter, so fit2’s
normal(2, 2)ona ~ Ageputmu = 2 + 2 Agebelow 0 onGamma("identity"). After such a failure the fit restarts from the placed intercepts with the slopes at zero; with fit2’s priors the update of ledger rowbrmsfit-methods:955then converges near brms’s posterior means. A fit that converged from its start is never restarted.emmeans()stopped with “undefined columns selected” when a transformed predictor’s variable was found in the formula environment rather than in the data.vcov_cluster()andcluster_scores()stopped with “A map factor length must equal parameter length” on every ordinal fit whosediscis held at 1.confint(parm = "Intercept[2]")on a family that holds that threshold through a transform says so and points tofixef()andhypothesis().emmeans()on a fit with a transformed predictor (poly(z, 2),log(abs(z) + 1),scale(z)) stopped with “undefined columns selected”. The reference grid now holds the variable at its mean, as brms’s andlm()’s do, and the transform is evaluated with the fit’s frozen basis, soscale()takes the training center and scale andpoly()its training coefficients, also underat =. The means agree with brms 2.23.0 at fixed parameters and withglm().conditional_effects()on agp()fit had nothing to draw.y ~ gp(x)stopped with “No plottable predictors found”; agp()term’s covariates and itsbyvariable are now effects, and a term over several variables adds their two-way displays, as brms’sget_all_effects_type(x, "gp")does:x,fandx:fforgp(x, by = f).fitted()’sEst.Erroron a category distribution left out an exactgp()’s kriging variance at a position the fit did not see, and a new grouping level’s variance, which the scalar route has always added. Past the data the shortfall was 8 percent of the standard error on thetest-fd-chain.Rfixture. The analytic routeconditional_effects()takes for an ordinal family had the same gap.fitted()’s finite-differenceEst.Errorcosts one pair of evaluations per smooth orgp()block where it cost one per coefficient. Such a block reaches a category probability only through its predictor’seta, linearly, so the derivative is the chain rule(d p / d eta) Z. Counted calls of the fitted value ondev/gpby-fdcost.R’s cells: a 160-coefficientgp(x)at three new rows 330 to 12,s(x, k = 8) + (1 | g)26 to 16 and 24 to 14, a 12-columngp(x, k = 12)34 to 12, a control unchanged at 12.Kriging at an unseen
gp()position works in correlation units, so a draw whose standard deviation underflows no longer stops the solve as exactly singular, and a row at an observed position is the fitted value exactly rather than through the kriging weights.A gradient that is not finite at the reported optimum passed
check_convergence()in silence, because its test compares a finite number withgrad_tol. nlminb reports X-convergence there (a three-component ordinal mixture whose probit component saturated, gradientInf). The fit now warns and names the parameters.The draws of a model whose location predictor has no column (an ordinal
y ~ 1with a modeleddisc, ory ~ cs(x)) were named one column off:brms_par_labels()dropped the labels of every distributional coefficient, so frmtmb.sample stored disc’s slope asb_Intercept[1]and the last threshold astau_raw[3](dev/ordmix-emptybeta.R).?refitsays thatrefit()does not repeat the warningfrm()gives when a nonlinear model’s likelihood is flat at the optimum: the design and its flat direction are the original fit’s.vignette("brms-migration")was stale onhurdle_cumulative()(it takes the threshold structures,thres(gr = )andcs()), onacat()’s links, ondisc, and onorder = "mu"for ordinal mixtures, and now says whatcs()on the cumulative families does.
Extension API
-
ord_thres_linpred()joins the sampling API: brms’s linear predictor with the thresholds included, at a fit’s estimates, which frmtmb.sample’sposterior_linpred(incl_thres = TRUE)reads per draw. An ordinal mixture is refused first, in brms’s words: “‘incl_thres’ is not supported for mixture models.” # frmtmb 0.67.0
Three lanes of brms parity, each with an adversarial review and punch rounds: ordinal (brms’s disc, threshold structures and acat() links on every ordinal family), ceplot (plot() of conditional effects and hypotheses, new levels in conditional_effects() and fitted(), small post-fit methods) and formrobust (addition-term expressions, update() that keeps the parameter formulas, bernoulli coding, the autocorrelation functions, offsets in the grids). Each lane’s dev/<lane>-findings.md has the validation, the numbers and the scripts, and dev/reviews/2026-09-30-<lane>.md has the review; dev/round-20260930.md is the round’s record. In brms’s own ported suite, bin 1 now passes 387 of 494 assertions, up from 368 at 0.66.0.
Breaking changes
Every ordinal family has brms’s discrimination
disc.cumulative(),sratio(),cratio()andacat()takelink_disc = "log"andthreshold = "flexible", in the order of brms’s constructors, anddiscis held at 1 unless the formula models it:bf(y ~ x, disc ~ 0 + z).lf(disc ~ ...)is no longer refused, andfamily()listsdiscamong the dpars. Held at 1,discshows nowhere, as in brms: not on theLinks:line, not as aFixed dpar:, not infixef(flatten = TRUE)orcoef(). Every other output of a flexible fit is unchanged: on 26 models of every kind, 570 of 581 outputs areidentical()to frmtmb 0.66.0, and the 11 that are not are the changes listed here (dev/ordinal-p1-bitwise.R). The likelihood cannot tell an intercept indiscapart from the scale of the thresholds, so the fit warns unless a prior holds it, also when cell-means columns (disc ~ 0 + h) add up to one; brms holds it with its defaultnormal(0, 1).hurdle_cumulative()hides itsdischeld at 1 too, as brms does: itssummary()andprint()no longer readdisc = logandFixed dpar: disc = 1, andfixef(flatten = TRUE), and with it the default statistic offrm_bootstrap(), drops the constantdisc_(Intercept)entry.fixef(flatten = TRUE)names are again all row names ofconfint(), the invariant?fixefshows.default_prior()lists oneInterceptrow per level ofthres(gr = ), as brms does, beside the class-wide one; each is a rowset_prior(group = )takes.acat()takes every link brms takes for it:probit,probit_approx,cloglog,cauchitandsoftitbesidelogit. Off the logit, a category probability is brms’s second form, a product of distribution functions times a reversed product of survivals, normalized (brms:::inv_link_acat()). They were refused.plot()of aconditional_effects()result and of ahypothesis()result returns plot objects, invisibly, as brms’s methods return their list of ggplot objects: a named list offrmtmb_ce_plotobjects (one per effect) or a list offrmtmb_hyp_plotobjects (one per page). Each draws when printed or plotted. Both methods returned their input before. A hypothesis plot stacks up tonvariables = 5hypotheses on a page, as brms’s does, where it drew one page per hypothesis, and a surface is drawn as colored contour lines (brms’sstype = "contour") where it was an image with contours. Graphical parameters (col,main, …) are ignored as before, and now with a warning that names them.predict(sample_new_levels = "old_levels")chooses one seen level per grouping factor, as brms’sget_new_rdraws()does, not one per term. Onbf(y ~ x + (1 | g), sigma ~ (1 | g))or(1 + x || g)a new group took each term’s effects from a different seen group, a group the data does not have. Undergr(g, by = f)the choice is made among the row’s own by-level only, with no draw for the by-level the row does not read. A multivariate call makes one choice for all its responses, so a grouping factor two responses share reads one seen group in both, as in brms.conditional_effects(effects = )checks each effect against the model, as brms does. An effect naming a variable the model does not read is dropped with brms’s warning “Some specified effects are invalid for this model”; with no valid effect the call stops with brms’s “All specified effects are invalid for this model”, which lists the valid variables. One invalid effect among valid ones was an error, and a variable in the data but not in the model drew a flat curve.update()with a complete formula keeps the stored parameter formulas, as brms’supdate.brmsfit()keeps them throughupdate.brmsformula().update(fit, y ~ x2)onbf(y ~ x, sigma ~ z)fittedy ~ x2with a constantsigma; it now fitsbf(y ~ x2, sigma ~ z). A formula of the update replaces a stored one of the same name, with brms’s message. On a nonlinear fit the update replaces the body and keeps the nonlinear parameters’ formulas, with brms’s message “Argument ‘formula.’ will completely replace the original formula in non-linear models”, soupdate(fit, bf(count ~ a + b, nl = TRUE))now refits where it was refused (ledger rowbrmsfit-methods:955). The updated fit’s call holds the pooled formula as afrmtmb::bf()call; the family goes in as its constructor call only when that call says everything the family holds, and as the family object otherwise, so an option such ashuber(k = 3)survives the update. A complete formula on a multivariate fit is refused, as brms refuses it (“Updating formulas of multivariate models is not yet possible”); it replaced every response’s formulas before.bf(nl = TRUE)without a parameter formula is no longer refused atbf();frm()refuses it, as brms refuses it atbrm(). The formulas may arrive afterwards,bf(y ~ a * x, nl = TRUE) + lf(a ~ 1), or from the fitupdate()keeps.A
bernoulli()response can hold any two values. They are coded 0 and 1 by level order, as brms codes them (-2,-1become 0, 1; so do1,2,0.1,0.9,"no","yes"andFALSE,TRUE). The coding is kept with the fit and applied to the response of newdata inresiduals(),pp_check()and frmtmb.sample’spredictive_error(). Before, anything but 0/1 was refused (ledger rowstandata:75). Two values that both lie strictly between 0 and 1 are coded too, with a warning that they look like a proportion, whichbinomial()withtrials()orBeta()fits (lme4#682); effect coding such as -0.5 and 0.5 does not warn.refit()takes the 0/1 codes thatsimulate()returns, and refuses any other value by name.refit()of any family refuses a response holdingNA. One divergence: brms codes an all-TRUElogical response 0, and frmtmb codes it 1.An addition term’s expression that gives
NAis refused by name, as brms refuses it atstandata():trunc(lb = ifelse(x > 1, NA, -5))andtrials(ifelse(x > 1, NA, n)). At 0.66.0 the expression was a column of the model frame, andna.omitdropped those rows.An addition term reads data columns only (user decision, 2026-09-30), as brms requires. A bare name such as
weights(w), withwa full-length vector outside the data, worked at 0.66.0 and is now refused with a message that namesw. Predictor formulas and offsets keep R’s rule of looking in the formula environment. See “An addition term takes an expression” under Formula grammar.
New features
threshold = "equidistant"andthreshold = "sum_to_zero"oncumulative(),sratio(),cratio(),acat()andhurdle_cumulative(), which refused both before. Equidistant thresholds are the first threshold plusk - 1timesdelta, as in brms;variables(),summary()andhypothesis()name itdelta(delta_<k>per level ofthres(gr = ),delta_<resp>in a multivariate model), andset_prior(class = "delta")gives it a prior.deltais positive for the two families whose thresholds are ordered, as brms bounds it. Class"Intercept"then addresses the first threshold alone, as in brms. Sum-to-zero thresholds sum to zero and the design is not centered, as in brms. brms declares one parameter per threshold there and relies on its prior for the common location, which the likelihood does not see; frmtmb estimates the free directions instead, so class"Intercept"has no parameter to address and is refused with the reason. Both structures work withthres(x = ),thres(gr = ),cs(),disc, the post-fit methods and the one-step residuals.equidistantneeds two thresholds per vector, sincedeltahas nothing to measure on one.sum_to_zeroholds a single threshold at zero and is fitted there, with no threshold parameter at all: underthres(gr = )brms runs the same model, and on an ungrouped two-category response it is the logistic regression without an intercept.default_prior()listsdeltawith brms’s lower bound of 0 for the ordered families.The log density with
disc, with both structures and ofacat()off the logit is brms’s: against brms 2.23.0’s compiled Stan program at 29 shapes, the value agrees to at most 2.6e-16 relative and brms’s gradient at frmtmb’s optimum is at most 2.3e-4 (rows 12e to 12h oftest-brms-likelihood.R;dev/ordinal-findings.mdlists the shapes brms’s own program fails on).plot()of a conditional-effects object takes every argument of brms’splot.brms_conditional_effects():rug,mean,stype,jitter_width(with brms’s deprecation warning), the layer listsline_args,cat_args,errorbar_args,surface_args,spaghetti_args,point_args,rug_argsandfacet_args,theme,askandplot. frmtmb draws with base graphics and has no ggplot objects to return, soplot = FALSEreturns the plot objects undrawn. The layer lists are translated to base graphics (colour,fill,alpha,linewidth,linetype,shape,size, an error bar’swidth, a rug’ssides, a surface’sbins, a facet’sncol,nrowandscales), and an element with no counterpart is named in one warning. A ggplot2themeis accepted with a warning that it is not applied.points = TRUEon aconditional_smooths()result is brms’s error.plot()of a hypothesis takes every argument of brms’splot.brmshypothesis():nvariables(and its aliasN, with brms’s warning),ignore_prior,chars,colors,theme,askandplot. A fit has no prior draws, soignore_priorchanges nothing, as it changes nothing in brms without them.fitted()takessample_new_levels = "old_levels". Each unseen level reads one seen level of its grouping factor, chosen at random once per call as brms chooses it and aspredict()chooses it, and every term of the factor reads that level: the row takes its fitted effects, andEst.Errortheir conditional variance. One choice serves every response of a multivariate call. At seeds 1 to 12 brms 2.23.0 and frmtmb choose the same level on(1 | g)with one or two unseen levels, on(1 | g) + (1 | h), on(1 | g)inmuandsigma, and within the row’s by-level ofgr(g, by = f)(dev/ceplot-oldlevels.R,dev/ceplot-rev-oldlevels.R).parnames(), brms’s deprecated spelling ofvariables(), answers on a fit with brms’s deprecation warning, once, whether or not brms is loaded. The generic moves from frmtmb.sample to frmtmb, so one generic serves fits and draws.conditional_effects()draws a new level where it refused to: crossed(1 | g) + (1 | h) + (1 | g:h)withgandhat observed levels the data never has together, underband = "boot"and on draws. The bootstrap band has the Wald band’s width (0.89 to 1.08 at 200 refits, seeds 5 to 7,dev/ceplot-rev-bootratio.R). With one new level in the call, the band on draws equals brms’s at the same draws and seed to 4.4e-16 (dev/ceplot-crossed-brms.R); with two, brms draws level by level and frmtmb draw by draw, so the bands agree in law and not number. A new member of anmm()term with abyvariable is drawn in its own by-level’s block: the bootstrap band is 0.83 to 1.11 of the Wald band’s width at 200 refits (the same script). brms 2.23.0 cannot draw such a level at all; it stops with one of three errors.conditional_effects(categorical = FALSE)on an ordinal fit gives brms’s warning “Predictions are treated as continuous variables”. The default per-category display gives none. This is a deliberate divergence from brms, decided by the user: brms’s default display is the expected category number and warns on every default call, and its own warning calls that display likely invalid for ordinal families and asks forcategorical = TRUE, which is frmtmb’s default.
Formula grammar
An addition term takes an expression, as brms evaluates it on the data rows:
weights(wt * 2),rate(time * 2),se(s / 2),trunc(lb = lb - 1),subset(x > 0),index(id * 3). Before, the expression went into the model frame as a formula term, wherewt * 2is an interaction, and the fit stopped with “invalid model formula in ExtractVars”. A single value is used for every row, sotrunc(lb = min(y) - 1)works too, and a value of any other length is refused by name. Fit,predict(),fitted(),simulate()andconditional_effects()read the expression the same way, in a multivariate model too. Every variable an addition term reads must be a column of the data, as brms requires:trials(k),trials(k + 0),weights(wt * k)andrate(time * k)withkdefined only outside the data are refused with a message that nameskand asks for it in the data (0.66.0 refused them too, as a scalar that is not a column, or stopped with “invalid model formula in ExtractVars” where the expression was not a valid formula term). Predictions on newdata and refits read the term again, and an outside value can be missing or changed by then, for example aftersaveRDS()andreadRDS()in another session; intrials()that change would be silent. Breaking for one case 0.66.0 read: a bare name such asweights(w)withwa full-length vector outside the data is now refused too. Function calls such aslog()are allowed, and predictor formulas and offsets keep R’s rule of looking in the formula environment. A name that R finds only as a function, astinweights(t * 2)without a columnt, is refused by the same rule. A prediction reads an addition term from newdata alone. The interval bound ofcens(x, y2)takes one value per row, as brms requires.weights(w, scale = TRUE)scales the weights to a mean of one on the rows the response reads, as brms does.brms’s deprecated
0 + interceptis accepted with brms’s warning (“Reserved variable name ‘intercept’ is deprecated”) and read as brms reads it, a data column of ones whose coefficient isintercept. Newdata needs no such column (ledger rowsstandata:970,:974).ar(),ma(),arma(),cosy()andunstr()exist as functions that return the term, as in brms. Added to a formula,bf(y ~ 1) + arma(x), the object is refused with brms’s message (ledger rowbrm:112). brms’s two ways to write the terms apart from the formula work:bf(y ~ x, autocor = ~ ar(t, g))andbf(y ~ x) + acformula(~ ar(t, g)).
New arguments and answers
frm(drop_unused_levels = ), brms’s argument. WithFALSEan unused level of a factor predictor gives a column of zeros, which frmtmb drops with its rank-deficiency message (ledger rowstandata:1133: brms keeps the column).autocor()on a fit, brms’s deprecated accessor, returnsNULLwith brms’s deprecation warning, as brms does for every fit since 2.11 (ledger rowsbrmsfit-methods:112,:113).predict(newdata = )underar()/arma()withcov = FALSEanswers when the response isNAin some rows, or absent. As in brms’s.predictor_arma(), the recursion runs in each group’s time order and a missing response is filled with a draw from the family at its shifted mean, so the rows after it read that draw’s residual. Before, the call was refused by name (ledger rowbrmsfit-methods:747).fitted()diverges from brms here, on purpose: brms fills with a draw per posterior draw, so itsfitted()carries the fill’s spread, and a maximum likelihood fit has no draws to carry the fill through.fitted()fills with the expected value, which is the mean of brms’s fill at fixed parameters, and itsEst.Erroris the parameter uncertainty alone (see?fitted.frmtmb_fit). frmtmb.sample’sposterior_epred()fills with draws, as brms does.
Bug fixes
A
hurdle_cumulative()fit’s draws are namedb_Intercept[k]in frmtmb.sample, as the other ordinal families’ are: the family now declares its inverse threshold map. They weretau_raw_k.conditional_effects()of a response whose predictor readsmi(x, idx = )stops with the reason: the grid holdsidxandindex()at one reference value, so every grid row names the same row ofx. It stopped with “Could not match all indices” from inside the prediction. brms 2.23.0 stops on the same grid.A grouping term
(1 | g:h)on integer or character columns predicts on new data.g:hwas evaluated as R’s sequence operator, so everypredict(),fitted()andconditional_effects()on new data stopped with “non-conformable arrays” (and a “numerical expression has 3 elements” warning escaped), or reported observed combinations as new levels. It is now the interaction the fit reads.conditional_effects()varies a variable the model reads through a transform,xinpoly(x, 2)orzinlog(abs(z) + 1), as brms does. The model frame keeps only the transformed columns, so naming such a variable stopped with “not stored in the model frame” and the default display found nothing to draw. The fit keeps the raw variables beside the model frame for it.conditional_effects()andemmeans()on a model with anoffset()stopped with “non-numeric argument to mathematical function” and “undefined columns selected”: the model frame held the columnoffset(log(time))and nottime, so the grids had notime. The frame now holds the offset’s variables, and both grids hold them at their means, as brms’s do.emmeans()includes a predictor’s offset at the grid’s value,log(mean(time))or the valueat =sets, which is emmeans’s own.offset.column and what brms’semmeans()gives. An offset inside a nonlinear parameter’s formula is left out of the body’s emmean, as brms leaves it out, andepred = TRUEincludes every offset once. One divergence, on purpose: over the responses of a multivariate fit with noresp =, each response gets its own offset, where brms’s single.offset.column adds one response’s offset to every response. An offset’s variable is not a default display ofconditional_effects(), as in brms, and a factor read inside an offset, as inoffset(log(as.numeric(f))), no longer lets model.frame()’s “is not a factor” warning escape.
Extension API
-
arma_cond_fill_dpars()gives a response’s parameters on newdata for a predictive draw, filling a missingcov = FALSEresponse as brms does,arma_cond_fill_epred()gives one draw’s expected response with that fill, andresponse_codes_newdata()codes a response read from newdata as the fit coded its own. All three are on?frmtmb-sampling-api, for frmtmb.sample.
frmtmb 0.66.0
Six lanes of brms parity, each with an adversarial review and one to three punch rounds: sampfix, fams2, aterms2, formula2, defects and postfit2. Each lane’s dev/<lane>-findings.md has the validation, the numbers and the scripts, and dev/reviews/2026-09-29-<lane>.md has the review; dev/round-20260929b.md is the round’s record. Silent wrong answers fixed: the display of an observed group under re_formula = NULL, the bootstrap of a model with a smooth, the linear predictor of an ordinal cs() fit, and the residuals of a missing mi() response. In brms’s own ported suite, bin 1 now passes 368 of 494 assertions, up from 275 at 0.65.0.
Breaking changes
-
frm_bootstrap()never redraws a smooth (user decision, 2026-09-29, withdrawing the whole-model default of 2026-09-24). It now readsre_formulaexactly aspredict()andsimulate()do. A smooth,gp()orhsgp()term, one indexed by a grouping factor included (s(g, bs = "re"),s(x, g, bs = "fs"), at2()with anremargin), is a term of the formula and not a group effect: every replicate holds it at its fitted coefficients and the refit estimates it again.NA,~0and~1redraw every true group-level effect,NULLholds all of them, and a one-sided formula holds the terms it names. The oldNAdefault redrew each smooth from its prior, which treated a smooth as a group effect, contrary to the package’s ownre_formularule, and turned the spread of the fitted curve into the spread of random curves. Ondev/simnewdata-boot.R’sy ~ s(x)(200 rows, 40 refits) the bootstrap SD of the smooth’s SD parameter was 1.4376 atNAand 0.3693 atNULL; it is 0.3693 at both now, and the replicates are identical (dev/postfit2-p1-simboot.R).confint(method = "boot")callsfrm_bootstrap()and changes with it on a model with a smooth.conditional_effects(band = "boot")inherits the rule. Its band ony ~ s(x)was 10.2 to 17.8 times the Wald band; it is 0.73 to 1.03 times now, the same underre_formula = NAandNULL, and 0.75 to 1.03 times ony ~ s(x) + (1 | g)at the population, at a new group and at an observed one, where the old band missed its own estimate (dev/postfit2-p1-smooth.R, 60 refits). nobs()is brms’snobs(): the rows of the data, after the rule for missing values and before anysubset()cut, andnobs(fit, resp = )one response’s rows. It refusedrespbefore. On a model withsubset()it is therefore not the count the likelihood sums over.logLik()’snobsattribute, whichBIC()reads, anddf.residual()keep that count, soBIC()is unchanged; a hand-written penaltyk = log(nobs(fit))on such a fit now uses the data’s rows, as it does on a brms fit.fitted(scale = "linear")on an ordinal fit with acs()term returns one layer per threshold,n x 4 x (K - 1)namedeta1,eta2, … as brms does; on a multivariate fit the layers stack with the other responses’, as in brms. It returned the shared predictor alone, which is the predictor at no threshold of the model, because acs()term moves each threshold’s predictor by its own coefficient. The layers are the ones the density reads: on ansratio()fit,P(Y = 1)isF(tau_1 - eta_1)of the first layer to within rounding.The prior table spells a smooth’s unpenalized column and a nonlinear parameter’s intercept as brms does,
sx_1fors(x).fx1andInterceptfor(Intercept), and lists a block’s coefficients in brms’s order, sorted by name.set_prior(coef = )takes either spelling.The ordinal families’ draws in frmtmb.sample carry brms’s names (
b_Intercept[k],bcs_<column>[k]); see its NEWS. frmtmb.sample needs this frmtmb for them.
New families
-
xbeta(),zero_inflated_beta_binomial()andhurdle_cumulative(), with brms 2.23.0’s parameters, links and defaults. Each reachesfitted()(brms’sposterior_epred()),predict(),simulate(),conditional_effects(),emmeans(),default_prior(),frm_compat()andfrm_sample(). The log-likelihood of each agrees with brms’s compiled Stan program at the same parameters to 1e-13, andzero_inflated_beta_binomial()agrees with glmmTMB’sbetabinomialwithziformulato 3e-9 in the log-likelihood.-
xbeta()is the extended-support beta for a response in[0, 1]with exact 0s and 1s. Its end masses are an incomplete beta function, taken from a continued fraction because the third derivatives ofRTMB::pbeta()areNaNat some ordinary points. WithRTMB::pbeta(), each of ten fits with a random effect stopped at “NA/NaN gradient evaluation”.RTMB::pbeta()is used only near the mean at large shapes, where its third derivatives are finite, and never at the mean itself, where all its derivatives areNaN. The result agrees withstats::pbeta()to 1e-12 (relative) for shapes from 1e-3 to 1e7, at the mean included. The interior density does not useRTMB::dbeta(), whose gradient isNaNpast a shape sum of about 1e3, soxbeta()fits precise data (tested atphi2e4, and at 2e5 withkappaheld). - When the response has no exact 0 or 1, only the interior shape places
kappa. Thexbeta()fit warns when it does not: whenkapparuns to 0 at every row, or when the standard error of a coefficient ofkappais above 10 on the log scale or not finite. -
hurdle_cumulative()codes its response0..K, with 0 the hurdle and the first level of an ordered factor.discis held at 1 unless the formula models it.thres(gr = ),cs()and athresholdother than"flexible"are refused. The likelihood cannot tell an intercept indiscapart from the scale of the thresholds. brms holds it with anormal(0, 1)prior; frmtmb warns unless a prior holds it.
-
-
cse(), brms’s second name forcs(), is accepted. -
beta_binomial()has a variance function, soresiduals(type = "pearson")works on it. It used to be refused with “has no variance function”. - The summary line of an ordinal family names the links of its other parameters, as in
cdf = logit; hu = logit; disc = log.
New addition terms
- brms’s addition terms
subset()andindex(), and the predictormi(x, idx = ). In a multivariate model,y1 | subset(s) ~ xfitsy1on the rows wheresisTRUE, so each response has its own rows, design, smooth bases and grouping levels. A missing value does not drop a row that every response using the variable leaves out, brms’sna_omit()rule.x | mi() + index(id)names the rows of an imputed response andmi(x, idx = ref)reads them by value, which brms requires when either response usessubset(); brms’s refusals are brms’s words. The log-likelihood equals brms 2.23.0’s compiled program at the same parameters, with and withoutmi(idx = )(test-brms-likelihood.R, rows 25 and 25b), and a subset model is the sum of the separate fits.fitted(),predict()andfrm_linpred()on such a model need oneresp =, as brms’s do, and onnewdatakeep the rows where the subset isTRUE. In a univariate modelsubset()filters the rows. See “Response subsets, subset() and index()” in?frmfor what is refused and the three differences from brms, all levels that occur only outside a response’s rows. - brms’s
rate()onpoisson(),negbinomial()andgeometric(): the expected count ismu * d, and the negative binomial’s shape is multiplied bydtoo, as in brms. Onpoisson()with the log link it is theoffset(log(d))model bit for bit. The log-likelihood equals brms’s compiled program on all three families (test-brms-likelihood.R, row 24).fitted(),predict(),simulate()and the residuals read the exposure;newdatamust hold it, and it must be positive there too, as brms requires. - brms’s deprecated
cat(K)is read asthres(K - 1), with brms’s warning;cat()besidethres()is refused.
Formula grammar
-
A distributional parameter can be equated to another, brms’s
bf(y ~ x, sigma1 = "sigma2"), also spelledlf(sigma1 = "sigma2"). In a mixture the components then share one parameter,sigma2;sigma1reads its coefficient, so the model has one parameter fewer and is the same function as the hand-written model with one shared sigma.fixef(),get_prior()andpar_template()holdsigma2alone, andvariables()andsummary()listsigma1beside it with its value, as brms lists its transformed parameter. brms’s five rules apply with brms’s messages;theta1 = "theta2", which stops brms with an internal error, is refused by name. brms’s log density at the maximum likelihood estimate equalslogLik()(test-brms-likelihood.R, row 17c). -
cmc, brms’s switch for cell-mean coding, onbf()andlf().bf(y ~ 0 + g, cmc = FALSE)keeps the factor’s contrasts (treatment, or polynomial for an ordered factor) and drops only the intercept column, in the population-level and the group-level terms, a multi-membership term included, with brms’s design and brms’s likelihood (row 24).re_formulanames such a term as it was fitted. It is refused onar1(),cs(),toep()and the distance structures where it changes a column, because they read one coefficient per level and have no brms counterpart; the refusal names the columns of both codings.lf(cmc = )was refused with “lf() takes two-sided formulas”, which was not the reason. -
y ~ .is expanded against the data, as brms does, infrm(),get_prior(),default_prior(),par_template()andfrm_simulate(). It was refused. Each parameter formula is expanded on its own, sosigma ~ .includes the response, as in brms (row 25). The fit keeps the expanded formula, soupdate(fit, newdata = )refits the same model, as brms’supdate()does. -
A list of families as
family =, one per response of a multivariate model, as in brms:family = list(gaussian(), poisson())(row 26). An entry fills only a response that has no family of its own, which is also brms’s rule for thefamilyargument. -
update()reads abf()delta as brms does:update(fit, bf(~ ., family = acat()))keeps the formula and changes the family, andbf(. ~ . + z, sigma ~ z)updates the location formula and adds the parameter formula. It was refused as a formula with no response. -
lf()takes whatbf()takes besides formulas, as brms’s does: a parameter fixed to a constant,lf(sigma = 2), or equated to another.
Post-fit functions
-
conditional_smooths()draws each smooth term (s(),t2()) on its own, which is brms’s function of that name, with brms’s grid, keys ("mu: s(x)","sigma: s(z)","mu_a: s(x)") and columns. On a fit,estimate__is the term at the estimate andse__its delta-method standard error over the term’s coefficients. A term of two numeric covariates is a surface, andtoo_fartrims it. Every smooth is drawn, a factor-smooths(x, g, bs = "fs")included, which is there_formula = NArule of 0.65.0. At brms’s parameter values (afixed_parambrms fit at the frmtmb estimate) the curve equals brms’s to 3.8e-15 ons(x),s(z, by = f)andt2(x, z), with identical rows, columns and attributes (dev/postfit2-brms-compare.R). The help page ofconditional_effects()said that function “also covers what brms callsconditional_smooths()”. It did not: it draws the expected response, which ony ~ f + s(x) + s(z, by = f)sits 0.0213 above thes(x)term with a standard error up to 1.214 times the term’s (dev/postfit2-before.R). The sentence is gone. -
make_conditions()andupdate_adterms(), brms’s helpers. Their output isidentical()to brms’s on 5 and 9 inputs.update_adterms()also takes abf()formula and keeps its other parts. -
conditional_effects()takes brms’ssurface,too_far,select_pointsandspaghetti.surface = TRUEwas refused before; it now draws a two-numeric effect over both ranges, andplot()draws the surface as an image with contours.select_pointsdrops the observations far from the values the curve holds the numeric predictors at. It measures the predictors only: brms also measures the RESPONSE against its mean, which hides the scatter the points are there to show (5 observations kept here where brms keeps 3 atselect_points = 0.1, the 2 dropped by the response alone).spaghetti = TRUEneeds draws, so on a fit it needsband = "boot", one curve per refit, and is refused by name with any other band. -
A nonlinear predictor gets the default Wald band.
conditional_effects()refused it before and namedband = "boot". The band is the delta method through the Jacobian thatfrm_lp_basis()tapes, which agrees with central differences to 2.1e-11 relative. Its standard error matched the standard deviation of 20000 curves drawn from the fit’s covariance: median ratio 1.0014 and 1.0008 over 100 grid points for two seeds, against a Monte Carlo error of 0.005 (dev/postfit2-nlmc.R). A display that runs through several predictors (the mean of a zero-inflated family, say), or a new level of a group-level term the displayed predictor itself reads, still needsband = "boot", and says so; a group only another parameter reads (sigma ~ (1 | g)) does not.
New arguments and answers
-
residuals()takes brms’snewdata, the response there minusfitted(newdata = )with that value’s standard error, andallow_new_levels. It answers a multivariate fit in brms’sn x 4 x nresparray, andrespnarrows it; it refused both before.type = "osa"stays univariate. -
ranef()takes brms’sparsandgroupsinstead of refusing them. -
family()takes brms’sresp;print()of a family takes brms’slinksandnewline, and a mixture prints as brms prints it,Mixtureand then each component. -
fitted()takessample_new_levels = "gaussian", which is what its interval at an unseen level already was;"old_levels"and"uncertainty"are refused by name, since this Wald summary has no draws of the group effects. -
predict()takessample_new_levels = "old_levels", which it refused: each unseen level borrows the effect of one seen level of its block, chosen at random once per call as brms chooses it, from every replicate’s draw of the group effects."uncertainty"is still refused. -
mixture()takes brms’sorder:"none"andFALSEare what a fit does,"mu"andTRUEare refused, and any other value is refused in brms’s words. -
categorical()reads a numeric response as categories in numeric order, as brms does. It claimed a response with ten categories had “fewer than two”. -
default_prior()andvalidate_prior()no longer read the response’s values, as brms’s do not (user decision, 2026-09-29): aBeta()model on a response outside (0, 1) gets its table, andfrm()still refuses the fit. An ordinal family is the exception, as in brms: its threshold rows are counted from the response, so a response it cannot read is refused. -
summary()reports agp()term’s standard deviation and length scale, under brms’s heading “Gaussian Process Hyperparameters” and brms’s row names,sdgp(gpx)andlscale(gpx), in$gp. The section had a printer and nothing to fill it, so neither number appeared in any summary. The length scale is on brms’s scale, the inputs divided by their largest distance;confint_varcorr()keeps it in data units. - A fit records the name of its data in
attr(fit$data, "data_name"), andupdate(newdata = )the new name, as brms does.summary()prints that name. A data frame passed by value throughdo.call()used to print its whole contents on theData:line, and prints nothing there now. -
update(fit, data = )stays accepted and refits on the new data, asstats::update(), lme4 and glmmTMB read it; brms refuses that spelling. A recorded divergence (user decision, 2026-09-29).
Bug fixes
-
conditional_effects()underre_formula = NULLnow reads each grid row’s groups by brms’s rule. brms predicts its display withallow_new_levels = TRUE: per grid row and per group-level term, the term reads its fitted effects where every one of its grouping variables is set to a level the fit saw, and a NEW level everywhere else. frmtmb used one rule for the whole grid: every grouping variable was a new group, even oneconditionsset to an observed level, and a variable varied as an effect ("x:g") was overwritten by a placeholder. Underband = "boot"every replicate then drew fresh effects into the first level, so levels 1 and 2 got the same band, bit for bit, and the band did not contain its own estimate. Found by lane sampfix on the draws method, which shares the construction. Now:- a row reads its groups’ fitted effects where it sets them to observed levels, in
conditionsor as an effect; - an unset variable, a level the fit never saw (
"99", which the frame now keeps instead ofNA) and a nested(1 | g / h)withgset andhunset each draw a new level, the last one withing’s own level; - rows that mix a level and
NAin one grouping column each get their own answer, where they stopped with “New levels … NA”; - under
band = "boot", a term read at an observed level is held at its fitted effects when the responses are simulated, and the other group-level terms are redrawn fresh (the smooths are held; see the breaking change above). A term read at an observed level in some rows and at a new one in others is refused by name.
On
y ~ x + (1 | g)with a group SD of 2 the refits’ mean at an observed level sat 0.857 modes from its estimate before and 0.089 after, and the boot band went from 3.117 to 0.998 times the Wald band (dev/postfit2-celevel.R). Only group-level terms enter this rule: a model with none reads nothing new underre_formula = NULL, so its band there is its band underNA. Reusing a bootstrap (boot = attr(ce, "boot")) for a call whose refits are simulated the other way is refused, saying so: a population call and an observed-level call can have the same grid.Two kinds of term read a row in their own way, and a third is refused:
-
gr(g, by = f)is one term per level off, and a row now reads only the term of its own level. The first build of this rule read every level’s term in every row, soband = "boot"and draws refused everyre_formula = NULLdisplay of such a model, even at an observedg. Withgunset or unseen, the row draws a new level with the SD of its ownflevel. On draws at brms’s parameter values, atg = "99"withf = "a"or"b", the curve and band equal brms’s (sample_new_levels = "gaussian", same seed) to 8.9e-16 (dev/postfit2-p2-brms.R). Withgunset, brms 2.23.0 stops (“missing sd_g__Intercept:f”); frmtmb answers as for"99". -
mm(g1, g2): the grid held unset members at their first level, an observed group, without a word. They are nowNAand read a new level, as brms reads them. Members with the same new value share one draw, and their weights add. At nothing set, the Wald band had the width of level 1’s curve (0.58 to 0.72); it is 3.73 to 3.75 now. The draws band equals brms’s at the same draws and seed (3.52 to 3.63). At an observedg1 = g2 = "2", the boot band was 5.1 to 6.0 times the Wald band, near the population curve, because the term was redrawn; the term is held now, and the ratio is 1.03 to 1.23 (dev/postfit2-p2-checks.R). A row with one member observed and one new is refused by name onband = "boot". Beside a plain(1 | g1), withg1unset or unseen, both terms now draw a new level onband = "boot"and on draws; they were refused. The new-level offsets have the model’s variance in both packages: 1.0085 against brms’s 0.9994, SE 0.013, over 12000 draws (dev/postfit2-p3-brms.R). - A factor-smooth, or a smooth with a
byfactor, set to a level that the fit never saw returned anNAestimate underre_formula = NULL, and underre_formula = NAa message that called the levelNA. Both now stop and name the variable, the level and the smooth. A grid that leaves such a factor unset (a(1 | g)besides(x, g, bs = "fs")underre_formula = NULL) stops too.
Reusing a bootstrap under another
re_formulais refused: on(1 + x | g)at an observedg, are_formula = NULLbootstrap was accepted by a~ (1 | g)call, whose estimate is a different curve. - a row reads its groups’ fitted effects where it sets them to observed levels, in
residuals(),pp_check()anddharma_residuals()no longer read a missingmi()response as an observation. The frame holds the placeholder 0 there, andresiduals()reported0 - muat those rows, where brms’s isNA. The residual there isNAnow, in all four columns, andpp_check()anddharma_residuals()leave the row out,pp_check()with brms’s warning. The univariatey | mi() ~ xhad the fault before; a multivariate fit refusedresiduals()before.residuals(type = "osa")on such a response is refused by name; it died inside TMB.A factor column of
newdataholding the levels as numbers (Trt = 0) is read as those levels, as brms reads it. It died with R’s “contrasts apply only to factors”. Prediction on a slice of the fitted data whose factor carries acontrastsattribute no longer warns “contrasts dropped from factor”.y ~ x + set_rescor(TRUE)and the other model-building helpers (lf(),nlf(),set_mecor()) written inside a formula are refused by name, in brms’s words. They died inmodel.frame()with “invalid type (list) for variable”.(cs(x) | g)is refused by name: on a family without category specific effects ascs()is, and on an ordinal family because a group-level category-specific effect is not supported. It died with R’s “could not find function "cs"”.se()on a family whose density does not read it is refused even when its column is missing from the data, as brms refuses it from the formula alone. The missing column was reported first and hid the refusal.
Extension API
-
?frmtmb-sampling-apiadds the engine ofconditional_smooths()(cs_build(),cs_coef(),cs_frame(),cs_finalize(),cs_probs()), brms’s display options (ce_spaghetti(),ce_spaghetti_check(),ce_check_distance()) and brms’s per-row, per-term rule for which group a grid row reads (ce_level_plan(),ce_plan_eval(),ce_plan_has_new()), which replacesce_new_level_spec(),ce_boot_grids()andce_draw_new_levels()in both methods. Those three are removed, withce_new_level_key(): nothing called them any more. An extension that did callsce_level_plan()andce_plan_eval()instead.ce_grids_build()takessurfaceandtoo_far,ce_frame()takessurface, andce_finalize()takesselect_points,pred_varsanddraws_by_eff. The effect frames carry brms’s"points"attribute. -
subset_resp_check()andsubset_newdata()join the sampling API (?frmtmb-sampling-api), for the draws methods of asubset()model. - The four built-in ordinal families and
thres(gr = )now declare the inverse of their threshold map,post$ord_thresholds_raw, beside the forward mappost$ord_thresholds.brms_fixef_rows()carries it for each threshold andcs()block. frmtmb.sample reads it to store an ordinal fit’s draws under the namesvariables(fit)gives them. An ordinal family from another package that declares only the forward map keeps its internal draw names there. -
?variablesno longer lists the ordinal andcs()exception for draw columns, which frmtmb.sample removes. It now says which draw columns are not brms’s: the covariance parameters, stored as sampled (theta_1).
frmtmb 0.65.0
Seven lanes, each with an adversarial review. Each lane’s dev/<lane>-findings.md has the validation, the numbers and the scripts, and dev/reviews/2026-09-29-<lane>.md has the review. Three silent wrong answers are fixed: cs() on a factor, re_formula = NA on a fit with a factor-indexed smooth, and the standard error of a category probability on any fit with a smooth.
Breaking changes
-
re_formula = NAnow keeps EVERY smooth, which is brms’s rule:re_formularemoves the(x | g)group-level terms and nothing else. Three kinds of smooth were dropped before, and each is indexed by a grouping factor:s(g, bs = "re")ands(x, g, bs = "re"), the factor smooths(x, g, bs = "fs"), and at2()with anremargin.brms::posterior_epred(re_formula = NA)is bitwise the same as atre_formula = NULLon a fit whose only group-indexed content is one of those, and differs ons(x) + (1 | g); frmtmb now agrees on both.(1 | g)is still dropped underNA, unchanged.This changes numbers, silently before and reported here:
-
predict(),fitted(),frm_linpred(),frm_lp_basis(),posterior_epred()andposterior_linpred()atre_formula = NAnow return the full prediction on a fit whose only group-level content is such a smooth. Ony ~ t2(x, g, bs = c("cr", "re"))the old answer was the intercept at every row. -
conditional_effects()passesre_formula = NA, so its curves change on those fits. Ony ~ s(x, g, bs = "fs")the drawn curve was FLAT, the whole fitted structure having been removed; it is now the curve of the grouping factor’s reference level, which is where the display holds every predictor it is not varying. -
emmeans()on such a fit now carries the grouping factor in its reference grid and averages the mean over its levels, where it used to drop the term.emmeans(re_formula = NULL)on the same fit works now too; it was refused for want of that column. -
simulate(re_formula = NA)no longer redraws those smooths, so the defaultpp_check()changes on any fit that has one. The draws were far more spread out than the model says: ony ~ s(x, g, bs = "fs")the per-row standard deviation of the draws over the fitted sigma was 2.93, and the pooled standard deviation over the data’s own was 1.34; both are 1.00 now. App_check()that looked mismatched on such a fit may now look right, and the earlier picture was the wrong comparison, not a finding about the model. - A smooth indexed by a grouping factor is kept, so
newdatamust carry its grouping column at everyre_formula, and a level the fit never saw is refused.re_formula = NAwas a way out of both and is not any more; the refusals say so. brms refuses both too, and not through its group-level machinery: the grouping factor of a smooth is an ordinary predictor there, so a missing column is a missing variable and an unseen level is “New factor levels are not allowed” whateverallow_new_levelssays. frmtmb keeps one opt-in brms does not have: underallow_new_levels = TRUEan unseen level of anfsterm takes the population curve, mgcv’sfsbasis returning a zero row for a level it does not know. Anrebasis or margin has one column per fitted level and no such row, so an unseen level there is refused either way. - To read the population curve off a model with an
fsterm, predict at an unseen level of the grouping factor withallow_new_levels = TRUE. The functional regression section ofvignette("case-studies")now draws its figure that way. - One fittable term is left with no
newdataroute at all: a factor smooth that also carries aby =factor,s(x, g, bs = "fs", by = f). mgcv’s random-effect split of that basis is one this version cannot invert, sopredict(newdata = ),fitted(newdata = ),frm_linpred(newdata = )andconditional_effects()stop with that named error at everyre_formula, wherere_formula = NAanswered before by dropping the term. What it answered was the intercept at every row, so a flat line became a named refusal, not a curve became an error. In-samplefitted()andfrm_linpred()on such a fit are unaffected. - A PARTIAL
re_formulaon a fit with such a smooth, for examplere_formula = ~(1 | f)ony ~ s(x, g, bs = "fs") + (1 | f) + (1 | h), is now accepted: it drops(1 | h)and keeps the smooth, as brms does. It was refused, becauseNAused to drop the smooth and a partial formula could not say which of the two it meant; with everyre_formulakeeping the smooth there is nothing left to decide.predict(),fitted(),frm_linpred()andsimulate()all take it. With it goes the last case of that refusal:?frm_linpredalso listed acar()orspde()field, but a field is a group-level term named by its grouping factor,(1 | loc), and was never refused. The help page now says so.
frm_bootstrap()is unchanged. Its default is a whole-model parametric bootstrap that redraws every block including the smooths, whichsimulate()does not offer. -
cs()on a factor or a character column was fitted and predicted on the factor’s INTEGER CODES, a silent wrong answer. The term is now expanded bystats::model.matrix()with the fit’s contrasts, so a discrete predictor becomes treatment-contrast dummies with one coefficient per dummy per category boundary, exactly as brms 2.23.0 builds itsXcs. What changes for a fitted model: a three-level factor gets FOUR coefficients where it had two, and their names move fromfc[1],fc[2](bcs_fc[k]) tofcb[1],fcb[2],fcc[1],fcc[2](bcs_fcb[k],bcs_fcc[k]), which is what brms calls them. Every estimate moves with them: on the recorded construction the log likelihood went from -446.9233 to -446.6361, the value the same model written with hand-built dummy columns already reached.newdatais recoded against the levels of the FIT rather than against its own, so a one-rownewdataholdingfactor("c")no longer returns the reference level’s probabilities, fromfitted(),predict(),simulate()andconditional_effects()alike, and a level the fit never saw is refused by name. A CHARACTERcs()column did not fit at all before:as.numeric()on it gaveNAs and the optimizer died on a NaN gradient.set_prior(class = "b")anddefault_prior()list and reach one row per dummy, under brms’scoefspelling.-
y ~ x + cs(x)is refused instead of fitted. The global coefficient of a column and the category-specific coefficients of the same column span one direction twice: adding a constant to the first and subtracting it from each of the others leaves the likelihood unchanged. It was fitted, and reported standard errors of 2.7e5 on every coefficient involved. The message names the edit that fixes it,y ~ cs(x), which fits the same set of distributions. A factor on both sides (y ~ f + cs(f)) and a constantcs()column are refused the same way. This is a deliberate departure from brms 2.23.0, which builds both blocks and samples the ridge: a Bayesian fit has a prior to hold the ridge and maximum likelihood has none.vignette("brms-migration")records the departure.The check is a rank test, so it catches a column that only a basis or an expression spells differently (
y ~ poly(x, 2) + cs(x),y ~ s(x) + cs(x),y ~ x1 + x2 + cs(x1 + x2), and twocs()terms that are the same column twice), and the message names the columns involved and the edit that works, which forpoly(x, 2) + cs(x)isI(x^2) + cs(x)and fors(x) + cs(x)is to drop one of the two. It does NOT reach anmi()orme()term, whose design column is a zero placeholder at assembly filled later with observed-or-latent values:me(x, sdx) + cs(x)appears to be identified andmi(x) + cs(x)is not settled.mo()has a placeholder column too, but whatever fills it is a monotone function of the predictor’s categories, somo()is tested against that category basis andy ~ mo(m) + cs(m)is now refused as well. It used to fit, buying 3 degrees of freedom for 6.4e-09 of log likelihood with NaN on all 9 standard errors.?frmstates themi()/me()gap.
The convergence check
-
The convergence warning “Large maximum absolute gradient at the optimum” no longer fires on correct fits. It used to read one absolute number,
frmtmb_control(grad_tol = 1e-3), against the largest gradient component, and that number does not mean the same thing on every design: the optimizer stops on a RELATIVE change in an objective that grows with the sample size, so the gradient at a good optimum grows with it too, and a parameter held on a bound byset_prior(ub = )has a nonzero gradient by construction. Over 720 correct fits on nine designs at four sample sizes the old check warned on 289 of them, rising from 11 percent of the fits at 200 rows to 75 percent at 20,000. It now warns on none of them.grad_tolis read twice: first as the gradient trip-wire it always was, then on the objective’s own scale. A fit that trips the trip-wire warns only if one exact Newton step over the parameters no bound holds would still gain more thangrad_tolin log-likelihood, and the warning reports that number, so it says how far short the fit is rather than only that a derivative is large. Every fit’s estimates, objective and optimizer status are bitwise unchanged; only the warning anddiagnose()differ. The warning names every number it used and the parameter each belongs to, so on a bounded fit it no longer prints the projected gradient under a phrase that says “maximum”, and it no longer sends the reader to the one parameter the verdict excluded.diagnose()gainsgrad_proj,grad_proj_par,grad_bound_heldandgrad_headroom, names the bounds that hold a gradient, and judges the gradient bygrad_tolinstead of a hardcoded1e-3that ignored the setting. On an importance-corrected fit it reports no verdict at all, becausefrm(importance = )optimizes a Monte Carlo estimate whose gradient no convergence criterion applies to, and there the clean verdict keeps being decided by the raw gradient againstgrad_tol, as it was before.The new criterion also reports what the gradient could not: a near-collinear design that nlminb calls converged at a gradient of
5.1e-3is short by 0.528 log-likelihood units, which the Newton step confirms to 0.2 percent and which a re-optimization from the same point does not recover. What it does NOT cover is documented on?frmtmb_control: the gradient is what admits a fit to the second reading, so a badly scaled column that holds the gradient down is never looked at, and theautoscaledefault is what closes that.
Refits and influence()
A refit inside the package now carries the FITTED model’s ordinal threshold count, and the per-level counts of
thres(gr = ), instead of recounting them from the data it is given.influence()andcooks.distance()rebuild the design from a subset, so deleting the last observation in a category used to refit a model with one threshold fewer: the deleted unit’s row of the table held anNA, its Cook’s distance wasNA, and the coefficients above the emptied category were reported one column early, with nothing but that trailingNAto show it. The deletion now refits the model that was fitted, and the row agrees with a fit that pins the count by hand.A refit also carries the fitted model’s response CATEGORIES. The model frame drops a factor level that no row of a subset takes, which renumbers every category above it, so an ordered-factor response could give a different influence table from the same data coded as integers. Emptying an INTERIOR category, and a hand-written
thres(K), were the two cases where it did. Integer, character and ordered-factor codings now give the same table, bitwise.influence()andcooks.distance()now count the deletion refits that failed. A partly failed table warns with the count and the first reason; a table whose every refit failed is an error instead of a silent matrix ofNA. Two refits cannot represent the fitted model at all and are refused by name:data =holding a response category, or athres(gr = )level, that the fit never saw, andgroups =deleting a wholethres(gr = )level.influence()warns when deleting a unit drops a coefficient. A deletion refit can succeed and still estimate fewer coefficients than the full fit, because the model frame drops an unused factor level: the last row of a rare level disappears from the subset. Those cells of the influence table stayNAandcooks.distance()isNAfor that unit, which it already did, in silence. The warning names the unit and the coefficients, and it fires for ANY factor level a deletion empties, not only acs()one:y ~ x + fwarns where it gave one silentNAbefore. Acs()factor level makes the case easier to meet, becausecs()gives each levelK - 1coefficients rather than one.frm_bootstrap(),refit(),frm_allfit(),anova(refit = TRUE),confint(method = "profile")and the autoscale pre-fit reuse the assembled design and already kept the count; that is now documented in?frm_bootstrapand pinned by a test.simulate()followed byfrm(), written out by hand, is not a refit and still counts from the data it is given, which?frm_bootstrapnow says.
Standard errors and the sampling API
-
A category probability’s
Est.Errornow carries the smooth’s uncertainty.fitted()on an ordinal, categorical or multinomial fit takes its standard errors by finite differences, and the coefficients it differenced never included a smooth’s. The differenced function was therefore “move the thresholds with the fitted curve frozen”, which is not the standard error of anything, while the analytic route used everywhere else has always carried the curve. Checked against Monte Carlo over the same joint covariance the delta method uses: ony ~ s(x, k = 8)withcumulative()the shipped value was 74 percent from the Monte Carlo standard deviation at the median cell, 0.37825 against 0.06509 on the first row, and is 2 percent from it now; on as(x, g, bs = "fs")fit it was 50 percent from it, and understated rather than overstated, because the cross term between the outer parameters and the coefficients can go either way. The same fix applies to the standard error of anautocor(cov = FALSE)quantity.What is left is first-order curvature, not a missing term, and it is the same on every earlier release: the corrected standard error is within 2 percent of Monte Carlo on a population smooth and within 8 percent beside a group effect, the worst measured cell being
s(x) + (1 | g)atre_formula = NULL. rescor_row_loglik()lost the Student-t joint density at a largenu. It wrotelgamma((nu + K) / 2) - lgamma(nu / 2)andlog(nu * pi)as they read, where the taped objective goes throughlgamma_shift_diff(). The as-written difference of twolgamma()values loses its leading digits asnugrows and overflows toInf - Infabovenu = 5.1e305, so the density left the gaussian limit it should approach and then stopped being a number. How far it drifted depends on the design, so no single figure is quoted here:dev/arcovsample-log/tnu.txthas one construction anddev/arcovsample-rev-log/13-rrl-ref.txtanother, with the loss reaching hundreds to tens of thousands of log units before theNaN. A sampler reaches those values whenevernuis weakly identified, sofrmtmb.sample’slog_lik()of aset_rescor(TRUE)Student-t model gaveNaNfor most draws. It now equals the objective’s ownmvt_std_loglik()at everynutried, and agrees with it to 2.8e-14 absolute over 200 NUTS draws whosenureaches 3e306.logLik()and the fit itself were never affected: those use the objective.Two exports for the sampling extension, documented with the rest of the contract on
?frmtmb-sampling-api:arma_cond_resp()names the responses carrying brms’s defaultcov = FALSEform ofar(),ma()orarma(), andarma_cond_dpars()gives aneval_dpars()list back with each such response’smumoved to brms’s one-step conditional mean. They exist so that the pointwise log-density of such a fit and the taped objective share ONE definition of the shifted mean rather than each holding its own;frmtmb.sample’slog_lik()reads them.
Other fixes
frm_simulate(prior = )used to fail insidevapply()with “values must be length 1” whenever one prior covered several parameters at once. It now says what it cannot do and names the alternative. This is about the shape of a prior entry, not one family: it is reached by class"Intercept"on an ordinal model with more than one threshold, by class"cor"on a block with more than one correlation, and by class"ar","ma"or"cortime"above order 1. The underlying defect, one draw written into every parameter the entry covers, is refused rather than performed; an entry covering exactly one parameter still draws.A
cov = FALSEar()/ma()/arma()group’s opening rows were described wrongly.?frmtmb-autocor, the comment inR/autocor.Rand the compatibility note all said a group’s first rows get no lagged term. Measured false: lagifirst reaches the row at within-group positioni + 1, so only the FIRST row of a group is unshifted, a row at positionkcarries the lags up tok - 1, and frommax(p, q) + 1on a row carries all of them. Measured overar(1),ar(2),ar(3),ma(2),arma(2, 2)andarma(3, 1)on groups of unequal length, one of them a single row (dev/arcovsample-log/firstrows.txt). The likelihood was always this; only the sentences were wrong.?frmtmb-medocuments whatme()returns, and a roxygen block inR/ad-env.Rno longer carries text after@noRd; both failed pkgcheck on the 0.64.0 release.
frmtmb 0.64.0
brms parity: every gap in the model menu that the last round listed is closed. Each lane’s dev/<lane>-findings.md has the validation, the numbers and the scripts that produced them.
New brms grammar
me(x, sdx)andme(x, sdx, gr = g)noise-free predictors, with brms’s meaning. The latent values are integrated by the Laplace approximation, which is exact for a gaussian response with linearme()terms. Several terms are correlated unlessset_mecor(FALSE)is added. The names are brms’s:bsp_mexsx,meanme_mex,sdme_mex,corme__mex__mezandXme_mex[i]. The log density agrees with brms 2.23.0’s Stan program to 1e-13 on five shapes. New data use the observed value of the noisy variable. brms’s prior classesmeanme,sdmeandcormeare refused by name.vcov(cluster = )now refusesme()fits, as its documentation already said, andanova()refuses fits whoseme()calls differ.thres()for the ordinal families.y | thres(K) ~ xfitsKthresholds, andy | thres(gr = g) ~ xfits one threshold vector per level ofg, with a count per level from the data or fromthres(n, gr = g). The log-likelihood equals brms’s compiled program’s to 17 digits on all four families, and agrees withMASS::polr()andordinal::clm()at the optimum. The thresholds have brms’s names,Intercept[g,k]andb_Intercept[g,k], andset_prior(class = "Intercept", group = "g")reaches one level.fitted()andpredict()give probability 0 to a category past a row’s own, as brms does.cs()andresiduals(type = "osa")are refused withgr. A threshold above the highest observed category is not identified without a prior, and the fit warns about it.gr(g, by = f)fits brms’s by-split group-level term: one set of standard deviations and correlations per level off, with each level ofgin exactly one level off. It is fitted as one block per by-level, with any structure whose density is a product over the grouping levels (us,diag,cs,ar1,toep, the spatial ones, anddist = "student"). Parameters take brms’s names (sd_g__Intercept:fa,cor_g__Intercept:fa__x:fa).ranef(),coef()andngrps()keep one entry over every level ofg, and a new level inpredict(allow_new_levels = TRUE)takes the covariance of its own by-level. brms’s refusals are reproduced word for word.mm(g1, g2, by = cbind(f1, f2))works the same way over the pooled levels.gr(g, by = f, cov = A)is refused by name, because brms correlates the by-levels throughA, which is not a by-split. It agrees with lme4 and glmmTMB fitted with one indicator term per by-level.ar(),ma()andarma()withoutcov = TRUEfit brms’s default residual-regression form, which was refused before.mugains a regression on the group’s earlier residuals and each row keeps the family’s own density, conditional on each group’s first rows. The likelihood matches brms 2.23.0’s to 1e-15 relative, for gaussian and student, withweights(),cens(),trunc(),mi(), random effects andrescor = TRUE. It also matchesstats::arima(method = "CSS")on one series. The coefficients are unconstrained, as in brms, and classarandmapriors and bounds act on them directly.fitted(),predict()andresiduals()use brms’s one-step mean, andsimulate()runs the recursion over its own draws. Other families are refused in brms’s words.0 + Interceptis brms’s reserved intercept:y ~ 0 + Intercept + xfits the modely ~ 1 + xwith the intercept as an ordinary class"b"coefficient, not centered. A class"b"prior reaches it and class"Intercept"is refused, as in brms. It works in the location, distributional and nonlinear-parameter formulas, factors take treatment contrasts, andnewdataneeds noInterceptcolumn.bf(center = FALSE)andlf(center = FALSE)are the same mechanism. Ordinal families refuse0 + Intercept, as brms does. The maximum likelihood fit is unchanged.
Multivariate models
Any number of
bf()formulas can be summed with+. The sum ofbf(y1 ~ x),bf(y2 ~ x)andbf(y3 ~ x)used to stop with “non-numeric argument to binary operator”, because the formula and the multivariate formula had two different+methods.lf()andnlf()take brms’sresp =to name the response they belong to.summary()of a multivariate fit prints brms’sFamily: MV(...)line and every response’s formula. A family added after several responses fills only the responses that have none; brms instead gives it to every response.student()withrescor = TRUEfits brms’s multivariate Student-t: onenushared by all responses (namednu, with no response), a sigma per response, and distributional sigma allowed. A formula or a constant fornu, and a mix of gaussian and student responses, are refused as brms refuses them. It agrees withmvtnorm::dmvt()and with brms’s ownlog_lik().predict()draws the responses jointly from the multivariate t.Ordinal responses in a multivariate model.
cumulative(),sratio(),cratio()andacat()responses keep their own thresholds (b_o_Intercept[1],set_prior(class = "Intercept", resp = "o")) and can share|ID|group effects with other responses. Other families with extra parameters (cox(),mixture_mvn()) stay refused there, now with the reason.fit_extras()takesresp, and the new sampling-API exportrescor_row_loglik()is the joint row density of a rescor fit.
Families
New families
hurdle_negbinomial()andzero_one_inflated_beta(), with brms’s parameters, links and defaults. Both reachfitted()(brms’s expected response),predict(),simulate(), pearson residuals,conditional_effects(),emmeans()andfrm_sample(). The hurdle negative binomial agrees with glmmTMB’struncated_nbinom2hurdle to 7e-11 in the log-likelihood, and both families agree with brms’s densities to a few ulps. A zero-one-inflated beta fit whose response cannot identifycoiwarns and names the remedy.BREAKING:
sratio()’s thresholds are unconstrained, as brms declares them, and no longer kept increasing. frmtmb held them as (first threshold, log increments), likecumulative(), so where the maximum has crossing thresholds the fit stopped on the ordering boundary with two thresholds equal, and was not brms’s mode. brms’s owninhalerexample withcs(treat)was one such fit: its log-likelihood rises from -455.115 to -451.257. Where the thresholds do not cross, a fit is unchanged to optimizer precision. The internaltau_rawof an sratio fit is now the thresholds themselves, so anewparamsorstartwritten for the old storage means something else. A class"Intercept"prior on sratio’s thresholds carries no Jacobian any more, as in brms and as oncratio().
emmeans
emmeans()takes brms’sdpar,nlpar,resp,epredandre_formula. A nonlinear model is supported:nlpar =averages that parameter’s linear predictor, and the wholemuorepred = TRUEuses the delta method throughfrm_lp_basis(). A multivariate fit withoutrespstacks its responses as brms’srep.measfactor. See?frmtmb-emmeans.Silent wrong answers in
emmeans(), now fixed:dpar =was ignored andmureturned;type = "response"did not apply the inverse link;s(),t2()andmo()terms were left out of the means;y | weights(w)was read as a response transformation;re_formulawas ignored.emmeans now shows frmtmb’s reason when it refuses, instead of “Perhaps a ‘data’ or ‘params’ argument is needed”.
Other fixes
A class
"Intercept"prior withgroup =on an ordinal model without grouped thresholds is refused. It used to be applied to every threshold without a word.residuals(type = "osa")on a zero-inflated, hurdle or zero-one-inflated family is refused by name. It used to fail with base R’s “comparison (==) is possible only for atomic and list types”.frm_compat()no longer callssimulate()refused forhurdle_poisson(),compois()andtweedie(), whose simulators work.summary()lists a grouping factor’s standard deviations before its correlations when the factor has several blocks, as brms does.
frmtmb 0.63.0
simulate()takesnewdata, andpp_check(newdata = )on a fit answers as brms does instead of refusing. The draws are for the new rows;pp_check()plots them against the newdata’s own response and readsgroupandxfrom it, as brms reads them. A newdata row with a missing response is dropped with brms’s warning, and a newdata without the response is refused for appctype, as brms refuses it. A grouping level the fit never saw is refused, as inpredict(), unlessallow_new_levels = TRUE, a new argument ofsimulate(); underre_formula = NAevery level is redrawn anyway, so an unseen one needs nothing. A residual correlation term such asar(time, gr = g, cov = TRUE)is rebuilt on the new rows, so rows of one group are drawn jointly at their lags, counted in the fitted time levels; a time the fit never saw is refused. brms counts a lag by a row’s position among the newdata rows of its group, so the two differ when newdata skips a time: rows at times 2 and 4 correlated at 0.389 here and 0.656 in brms on one AR(1) fit. The departure is deliberate, because under brms’s reading the correlation of two rows changes when a third row is added to newdata.mixture(groups = ),mixture_mvn(), and the hidden Markov and learning families of frmtmb.latent and frmtmb.learn refusenewdata: their draw walks the fitted groups or sequence, andmixture_mvn()goes through the same structured slot.BREAKING:
simulate()readsre_formulaaspredict()reads it.NULLkeeps every group-level term,NA,~0and~1keep none, and a one-sided formula keeps the terms it names. A term that is not kept is REDRAWN from its estimated distribution in every replicate, asNAalways did. Through 0.62.0 every formula meant “keep everything”, sosimulate(re_formula = ~1)gave the conditional draws whilepredict(re_formula = ~1)gave the population prediction.pp_check()anddharma_residuals()on a fit passre_formulatosimulate()and change with it. A term the fit does not have, and a partial formula on a fit with a factor-smooth term, are refused withpredict()’s words, and so is are_formulathat is notNULL,NAor a formula, which used to condition on everything with nothing said. When a formula keeps some columns of a term and drops others, as~ (1 | g)does on a(1 + x | g)fit, the dropped columns are drawn given the kept ones.BREAKING:
simulate(re_formula = NA)keeps a population smooth. It redrew the penalized coefficients ofs(),gp()andhsgp()terms from their smoothing prior, which replaced the fitted curve with a random one: ony ~ s(x)the draws spread with an sd of 2.13 around a fit whose residual sd is 0.28.pp_check()on a fit defaults tore_formula = NA, so its check of any smooth model compared the data with those draws. A factor-smooth term is group-level and is still redrawn.frm_bootstrap()is unchanged: by default it still redraws the group effects AND the penalized coefficients of every smooth,gp()andhsgp()term in every replicate, a whole-model parametric bootstrap, and its draws are identical to 0.62.0’s at the same seed.re_formula = NULLis the bootstrap that conditions on the fitted random effects and smooths and redraws the noise alone.?frm_bootstrapnow says which terms each setting redraws.The
pp_check(type = "error_binned")refusal on a multinomial fit calls the response a set of counts over categories, not a category.
Five defects filed in the 0.62.0 round (lane wt-predfix; dev/predfix-findings.md has each reproduction and measurement).
-
BREAKING: a fit with a badly scaled predictor is now standardized by default.
frmtmb_control(autoscale = )defaults toNULL, which runs theautoscale = TRUEpre-fit when a qualifying column has a standard deviation below 1e-3 and its coefficient is optimized directly, or when a column that carries a random slope ((1 + x | g)) has one below 0.05. Without it such a fit could stop short, report convergence 0 and say nothing: a poissony ~ 0 + xwithxon a 1e-6 scale sat 62.58 log-likelihood units belowglm(), the same happened with an intercept and a group effect in every family measured, by up to 91 units, and a random slope fell short by up to 14.4 units at a spread of 0.01.The pre-fit now also rescales a random slope’s column in
Z, with its log standard deviation and its effects, forus(),diag()and the Student-t blocks; before,autoscale = TRUErescaled the fixed effect only and a random slope on a 1e-6 column was still up to 6.35 units short. A random slope in any other covariance structure, or on a column the fixed effects do not carry, is not rescaled. The pre-fit’s own warnings are no longer shown; only the reported fit’s are.The default never reports less than
autoscale = FALSEwould. When the pre-fit errors, the default fits the model asautoscale = FALSEdoes. When the pre-fit stops without converging in its coefficients (a completely separated fit does), the default fits both from the pre-fit and without it and reports the one without it, with its warnings, unless the other has the better likelihood and either warns at least as often or is a verified optimum.verbose = TRUEsays which. Withautoscale = TRUE, which you asked for, a pre-fit error stands and namesautoscale = FALSE, and a pre-fit that did not converge warns.A
mucoefficient underREML = TRUEorprofile = TRUEdoes not engage it by itself: those coefficients are fitted by the inner solver, which reached the optimum at every scale measured. A fit that engages is not guaranteed to be higher: on 108 skew-normal fits it was never lower by more than 1e-6, and one REML random-slope fit was lower by 5.5e-7, which is optimizer tolerance. A fit with no such column is unchanged to the last bit: 444 of 480 fits over 32 families and five designs, under ML, REML andprofile = TRUE, are identical, and the other 36 differ only in thevcov()fix below.autoscale = FALSEturns it off, andfrmtmb_control()$autoscaleis nowNULL.diagnose()andverbose = TRUEsay when it ran. predict()now draws acs()term. On ansratio(),cratio()oracat()fit with a category-specific effect,predict()drew every row as if the term were absent, in sample and atnewdata, with no warning: atx = 3, where the category probabilities are (0.95, 0.002, 0.045), it reported about (0.42, 0.39, 0.19). The offsets were written one level too deep in the list the simulator reads, and atnewdatathey were the training rows’ offsets. Everypredict()of such a fit changes.fitted(),simulate()and the log-likelihood were right and are unchanged.cs_offsets_add()joins the extension API: it adds one response’scs()offsets atnewdata. The documentation ofwith_cs_offsets()now says what it does: it takes the list of every response, not one response’s list.fitted()answers a multivariate fit, in brms’sn x 4 x nresparray. It refused before, whilepredict()answered. A scalar response is one layer named by the response, and a categorical one is one layer per category,P(Y = k), as brms 2.23.0 stacks them.resptakes several responses in the order given, and one scalar response is still ann x 4matrix.fitted()on aquadrature = TRUEfit warns thatEst.Errorleaves out the group-effect uncertainty at a level the fit saw. The ordinal route andpredict()already said so; the ordinary route returned the smaller number silently. At a level the fit never saw nothing is left out, and it does not warn.vcov()answers aREML = TRUEorprofile = TRUEfit that has no random effect and no free dispersion (poisson, bernoulli, binomial, geometric, exponential, categorical). It died with “length of ‘dimnames’ [1] not equal to array extent”. The standard errors areglm()‘s for poisson (1.9e-11 relative) and bernoulli (3.8e-8). For geometric and exponential they differ fromglm()’s by 2.3e-2, because frmtmb uses the observed information andglm()the expected information, which differ on those families’ log link.A new grouping level refused by
predict(),fitted()and friends is now classedfrmtmb_new_levels, so an extension can say what it can do instead. The message is unchanged.BREAKING: in a multivariate model a prior needs
resp, as it does in brms. A class"b"or"Intercept"prior, a distributional parameter’s own class ("sigma","nu", …), and a residual correlation class ("ar","ma","cosy","cortime") written withoutrespare now refused with afrmtmb_error. Until now frmtmb applied such a prior to every response that had the slot, where brms 2.23.0 refuses it. The message gives theset_prior()call for each response on which that call resolves. What stops working:set_prior("normal(0, 1)", class = "b")onbf(y1 ~ x) + bf(y2 ~ x), and on ami()model, which is multivariate. Writeset_prior("normal(0, 1)", class = "b", resp = "y1") + set_prior("normal(0, 1)", class = "b", resp = "y2"). Class"sd"got the same rule in 0.62.0.BREAKING: where the location is several distributional parameters, a prior names one with
dpar. Oncategorical(),multinomial()andmixture()this is brms’s rule. frmtmb applies the same rule to its other families with several location dpars:mixture_mvn(), and in the extensionslca(),hmm(),lba()andrdm(). Class"b","Intercept"and"sd"withoutdparare now refused there; writedpar = "mub",dpar = "mu1"ordpar = "v1". Until now such a prior reached every category’s, component’s or accumulator’s coefficients at once.default_prior()lists these rows with theirdpar, where it listed them with none.Class
"b"reaches the category-specific coefficients ofcs()onsratio(),cratio()andacat(), andcoef = "x"names the termcs(x), as in brms. Until now class"b"missed them: onord ~ cs(x)it applied nothing, and onord ~ z + cs(x)it reachedzonly. A MAP fit with such a prior moves, because the prior now applies.default_prior()lists abrow for eachcs()term.BREAKING: on a nonlinear location a class
"b"or"Intercept"prior needsnlpar, as in brms. Until now class"b"withoutnlparonbf(y ~ a * exp(b * x), a ~ 1 + z, b ~ 1, nl = TRUE)went to the slopea_zand nowhere else, and on a model whose nonlinear parameters are all intercept-only it went nowhere, without a word. Class"Intercept"orcoef = "Intercept"reached the intercept of EVERY nonlinear parameter. All are now refused.class = "Intercept", nlpar = "a"stays accepted as frmtmb’s own spelling, although brms refuses it.BREAKING: three more spellings brms refuses are refused (user decisions, 2026-09-24).
respon a model with one response;respon class"rescor", which used to be ignored while the prior went on the whole residual correlation; and class"b"on a predictor with no population-level slope, such asy ~ 1, which used to apply nothing whileprior_summary()listed it.respon class"cor"stays accepted as frmtmb’s own spelling.A fit that used one of the refused priors does not move: it now stops before fitting. No other fit moves.
frmtmb 0.62.0
-
A
skew_normal()fit that used to stop atalpha = 0now finds the maximum.alpha = 0is a stationary point of the skew-normal likelihood at every sample, and the information is singular there, so a fit that reached it converged with code 0, no warning, and a log-likelihood up to 35 units below the optimum. The start is what put it there:alphawas started from the sign of the RESPONSE’s skewness, and a covariate can give the response the opposite skew from the residual. Two things change.alphaandsigmanow start from the response with itsmupredictor removed, including anyoffset(), not from the raw response. And a fit that lands onalpha = 0anyway is refit from+2and from-2, keeping the best optimum.Both halves read the
alphaPREDICTOR rather than an"(Intercept)"coefficient, so they apply toalpha ~ 0 + gas well as toalpha ~ g. Cell-means syntax used to get neither: it started every coefficient at zero, which is exactly the stationary point, and stayed there.mixture()carries a component’s escape through.That placement is confined to a dpar that declares a stationary point, which today means
skew_normal()’salphaand nothing else. Every other dpar keeps the previous rule, an intercept or nothing.Nothing is printed. The refit runs only when a fit reaches the stationary point, and on a 200-fit spread of skew-normal designs that was 10 fits, costing 0.1 extra optimizer runs per fit overall; a fit that pays for it takes about 1.3 times as long.
fit$opt$stationary_escaperecords the restarts and what they gained.What this changes for you, and nothing else does. A
skew_normal()fit moves when itsmupredictor is anything other than a bare intercept: a covariate, anoffset(), or a design with no intercept. It also moves whenalpha’s own design has no intercept. TherelogLik(), the estimates and the standard errors can all differ from 0.61.0, and where they differ the new value is the higher likelihood.A
skew_normal()fit withmu ~ 1and no offset is unchanged to the last bit, and so is every other family, including intercept-less designs such asy ~ 0 + gand distributional ones such assigma ~ 0 + g. A family’s
init_dparsfunction may declare a third argument,(y, aterms, resid), and is then given the response with itsmupredictor taken out by least squares. A two-argument initializer is still called with two arguments. A family may also declarepost$stationary, which says where its likelihood has a stationary point and where to restart from when a fit reaches one. See?frmtmb_family.
Seven places where frmtmb answered differently from brms 2.23.0 on the same model (lane wt-correct; dev/correct-findings.md has each construction and its brms measurement).
BREAKING:
residuals()refusestype = "response"(brms’s"ordinary", also the default) andtype = "pearson"on an ordinal (cumulative(),sratio(),cratio(),acat()) ormultinomial()fit, as brms refuses them (“Predictive errors are not defined for ordinal or categorical models”). Until now an ordinal fit returned a residual on the category codes,y - sum_k k P(Y = k), which brms does not define.type = "osa"still works on an ordinal fit and uses only the order.plot(fit)is unchanged.pp_check()on a fit works for every*_groupedtype and takesx.group = "g"andx = "x"reached bayesplot as one string, so every grouped type and every type with anxfailed. Both names are now looked up in the model frame, as brms looks them up, and a name the model does not use is refused. Every typebayesplot::available_ppc()lists (49 in bayesplot 1.16.0) now does on a fit what it does in brms, except theloo_*types, which need posterior draws and are refused with the reason.BREAKING:
pp_check()on a fit takes brms’s arguments in brms’s order:type,ndraws,prefix,group,x,newdata,resp, withre_formulaafter the dots. A positional fourth argument used to bere_formulaand is nowprefix.prefix = "ppd"plots the simulated responses alone. A type that is not a ppc type, such as"violin", is refused with the list of valid types instead of “object ‘ppc_violin’ not found”.type = "error_binned"is refused on an ordinal, categorical or multinomial fit, as in brms.newdata,draw_ids,nsamplesandsubsetare refused with the reason. On a categorical fit the simulated categories reach bayesplot as their codes, so"bars"and"error_hist"work there.pp_check()on a fit follows brms on two smaller points as well.respis accepted and ignored on a model with one response, which is what brms does with it there. And agrouporxgiven for a type that has no such argument reaches bayesplot, so bayesplot warns “unrecognized and ignored” exactly where brms makes it warn. brms is asymmetric about a name its data does not carry, and the asymmetry is copied: such agroupis dropped and warns nothing, while such anxbecomesnumeric(0), stays, and warns.BREAKING: a class
"sd"prior reaches only its own response, distributional parameter and nonlinear parameter, which is how brms scopes it.set_prior("normal(0, 5)", class = "sd", group = "g")onbf(y ~ x + (1 | g), phi ~ (1 | g))put the prior on thephiblock too, where brms leaves that block at its own default. In a multivariate model a class"sd"prior with noresp, and in a nonlinear model one with nonlpar, used to reach every block; both are now refused, as brms refuses them, and the message lists the prefixes the model has. A block that spans several predictors,(1 | q | g)inmuandsigma, keeps brms’s own rule: there a specification with a field left empty reaches the whole block. A fit with such a prior changes its estimate;default_prior()lists one class-wide"sd"row per prefix. This covers a SMOOTH’s smoothing standard deviation as well, which class"sd"also addresses here: onbf(y ~ s(x), sigma ~ s(z))a bare class"sd"prior reached both smooths and now reaches themuone only, anddefault_prior()grows adpar = "sigma"row for the other. brms keys itssdsrows by the same prefix.BREAKING: a
mi()coefficient is namedbsp_<resp>_mi<x>, as in brms, where it wasb_<resp>_mi<x>. This changesvariables(), draws, andhypothesis(): the bare"y_mixm > 0"is refused as brms refuses it; writehypothesis(fit, "bsp_y_mixm > 0", class = NULL).fixef(),vcov()andsummary()keep the namey_mixmand now list it after every ordinary coefficient, in brms’s order, with ami(x) * zmain effect ahead of its interaction.BREAKING: a mixture’s reference component is the one whose theta has no formula, as in brms.
bf(y ~ 1, theta2 ~ x)on two components used to be refused; it now fits with component 1 as the reference, sotheta2’s coefficients are the log odds of component 2 against component 1. A formula for fewer thanK - 1of the thetas is refused (“Can only predict all but one mixing proportion”), wheretheta1 ~ xon three components used to fit with an intercept-onlytheta2.?frmnow says, underREML, that mgcv’smethod = "REML"for a location-scale family integrates the coefficients of every linear predictor, so the two REML criteria differ for a smooth insigma.test-smooths.Rmeasures the gap.
predict() and fitted() carry the uncertainty in the group effects at a grouping level the fit saw, as brms does (user decision, 2026-09-22), and re_formula keeps the terms it names (lane wt-reunc). dev/reunc-findings.md has the construction and every measurement.
predict()draws the group effects at a known level. Each replicate draws the whole vector of group effects from its conditional law given the data and that replicate’s parameters, read from the fit’s joint precision. For a linear mixed model this is the prediction error variance of the best linear unbiased predictor, jointly with the fixed effects. Two rows of one group share that group’s draw, sopredict(summary = FALSE)is right across rows as well as per row. The interval at a known level is wider than in 0.61.0, where the effects were held at their modes and the interval under-covered. This holds under ML,REML = TRUEandcontrol(profile = TRUE).re_formula = NA, a model with no group-level term, and rows at a level the fit never saw give the same draws as 0.61.0, bit for bit. A fit made withquadrature = TRUEhas no group effects in its joint precision: it keeps them at their modes and warns.fitted()on an ordinal or categorical mixed fit carries the group-effect uncertainty inEst.Errorand in the interval. The finite-difference delta method used to cover the outer parameters only; the group effects now join it, with their covariance taken jointly with the parameters. On a cumulative fit with 10 groups the 0.61.0 standard error of a category probability was 0.59 to 0.95 of the Monte Carlo value over the same joint law, and forP(Y = 1)smaller than atre_formula = NA; it is now 0.92 to 1.09 of it. The scalar route offitted()already carried this term and does not change.-
BREAKING:
predict()’sparam_uncertaintyis nowpropagate_error, still defaulting toTRUE, and it now holds the group effects at their estimates as well as the parameters. The old name is removed rather than deprecated, so a call that uses it is an error naming it. The new name states the axis: whether the error in the ESTIMATES is propagated into the interval. A fitted group effect is an estimate, sopropagate_error = FALSEholds it, and what is left is the observation noise. The fresh effect drawn for a level the fit never saw is not an estimate and is unaffected, and neither is the observation noise itself.It is not
re_formulaand neither argument can say the other’s thing.re_formulachooses WHICH terms enter the prediction, so onlyre_formula = NAcan say “predict for an average group”;propagate_errorchooses whether their error is carried, so onlypropagate_error = FALSEcan say “include this group’s own effect but treat it as known”, which is what 0.61.0 did at every known level. brms has no such argument, because each posterior draw carries its own parameters and its own group effects.?predict.frmtmb_fithas the four combinations. BREAKING: a one-sided
re_formulakeeps the terms it names, inpredict(),fitted(),frm_linpred(),frm_lp_basis()and everything that reads them. It used to keep EVERY term with nothing said, sore_formula = ~ (1 | g)on a fit with(1 | g) + (1 | h)returned the full prediction. brms’s rule applies now: a term is kept when the fit has a term with the same grouping factor whose columns include its columns, so~ (1 | g)on a(1 + x | g)fit keeps the intercept and drops the slope. A dropped term is gone from the estimate, the standard error and the draws, and its grouping column is not needed innewdata.-
BREAKING:
re_formula = ~1means no group-level effects, as in brms and as~0already did, inpredict(),fitted(),frm_linpred()andfrm_lp_basis(). It used to keep all of them.simulate()is the exception and does not change. Therere_formulais lme4’s switch:NAREDRAWS the group effects and every other value conditions on them, so~1,~0and~ (1 | g)all still give the same draws asNULL. The two readings now differ, which matters fordharma_residuals()and forpp_check()on a fit, because both go throughsimulate(). What a partial formula should MEAN there is a decision rather than a fix, since brms has nosimulate(); it is filed indev/test-backlog.md. BREAKING: a
re_formulaterm that matches no term of the fit is an error that names the term and lists the fit’s terms. It used to be read asNULL. brms drops such a term silently, sore_formula = ~ (1 | nosuch)there is the population-level prediction; here it is refused, because a misspelled grouping factor should not change the answer with nothing said. Usere_formula = NAfor the population level.A
re_formulathat keeps SOME terms is refused on a fit that also has group-level content a formula cannot name, such as a factor-smooth terms(x, g, bs = "fs"):NAdrops that content andNULLkeeps it, and a partial formula cannot say which.
frmtmb 0.61.0
The silent wrong answers brms’s own ported test suite found (item 2.6f, lane wt-adefects). Each one is a case where frmtmb returned a number where brms refuses, or returned the wrong number with nothing said. dev/adefects-findings.md has the construction and the before-and-after figure for every item.
BREAKING:
ar(),ma(),arma(),cosy()andunstr()take ONE VARIABLE NAME as the time index. An expression was evaluated against the model frame and its values became the time points, soar(x + t, g, cov = TRUE)fitted a different likelihood and said nothing: log-likelihood -30.33421 against -30.88513 forar(t, g, cov = TRUE)on the same 24 rows. brms refuses the same call (“Cannot coerce ‘x + t’ to a single variable name”). Compute the column first and name it.time = NAand an omittedtimeare unchanged.-
BREAKING:
gr =takes variable names crossed by:and nothing else, which is brms’s grammar (“It may contain only variable names combined by the symbol ‘:’”). Two things change.gr = a:bnow CROSSES the two variables instead of being evaluated. R’s:is the interaction for two factors but the sequence operator for anything else, so on numeric group codesgr = g1:g2used to produce a length-one vector; it now groups by the pair, as it already did for factors and as it does in brms.Every other operator is refused.
gr = g1/g2on numeric codes used to group the residual by the QUOTIENT, merging the series (1, 1) and (2, 2), for a log-likelihood of -30.87554 against -30.88513 with nothing said.gr = g1 + g2,gr = g1 * g2,gr = factor(g)andgr = interaction(g1, g2)are refused too; the last two worked before and now need the column computed first, which is what brms asks for. BREAKING: a hypothesis must state a relation.
hypothesis(fit, "Age")used to be answered as the row of"Age = 0". brms refuses it, “Every hypothesis must be of the form ‘left (= OR < OR >) right’”, and so does frmtmb now. Write"Age = 0","x1 - x2 = 0"or a directional"x > 0". The bare spelling was documented as an extension; it is gone, and 48 call sites in this repository’s own suites were rewritten.A fit carries its model frame as
fit$data, which is what brms keeps there. Afrmtmb_fithad nodataelement, sofit$datapartial-matchedfit$data2and brms-shaped code read thedata2list (a covariance matrix, or an empty list) with no error. It is the same objectmodel.frame()already returned, so it costs no memory.predict(newdata = )may omit a grouping column whenallow_new_levels = TRUE, which is brms’s rule (“grouping factors do not need to be specified by the user if new levels are allowed”). The column is filled withNA, so every row is an unseen level: the population value, plus that block’s variance in an interval. It used to stop at base R’s “object ‘g’ not found”. Withoutallow_new_levelsthe call is still refused, now by a classed error naming the column and the two arguments that answer it.log_lik()on a maximum-likelihood fit says to sample, the wayloo()andwaic()already did. frmtmb defines the generic and thefrmtmb_fitmethod, and registers it on rstantools’ generic;frmtmb.samplere-exports it rather than defining a second one. It used to be “could not find function”.-
BREAKING:
predict()is brms’spredict()(item 2.6d). It returns a summary of the PREDICTIVE distribution of the response, with observation noise in it, in the columnsEstimate,Est.Error,Q2.5andQ97.5. It used to return the LINEAR PREDICTOR, glmmTMB’s convention, and on a lognormal fit the two differ by about 8 against about 6,700.What replaces it:
frm_linpred(), which is the oldpredict.frmtmb_fit()unchanged, under a name that says what it gives back. Every argument is the same:frm_linpred(fit, type = "link")is the oldpredict(fit),frm_linpred(fit, type = "response")the old response scale, andse.fit,dpar,resp,re_formula,allow_new_levelsand the glmmTMB aliasesconditional,zprob,zlinkanddispall behave as they did.predict()refusestype,se.fit,dparandscaleBY NAME and the message says where each one went, so a ported script stops rather than changing scale in silence.How the draws are made, since a maximum-likelihood fit has no posterior: each replicate draws the outer parameter vector from
N(theta_hat, vcov(fit, full = TRUE))and then a response from the family’s own simulator, the onesimulate()uses.param_uncertainty = FALSEdrops the parameter draw and simulates at the estimates alone. The random effects of a level the fit SAW stay at their conditional modes, as everywhere else in this package, so their own conditional variance is NOT in the interval;?predict.frmtmb_fitsays so anddev/shapes-findings.mdreports the measured coverage of each arm.ndrawssets the number of replicates (default 1000),summary = FALSEreturns them, androbust,probsandtransformare brms’s and are answered.draw_ids,coresandsort = TRUEare refused by name.A level the fit did NOT see is drawn, not held at zero:
allow_new_levels = TRUEtakes that level’s effect from its block’s own estimated covariance, once per replicate at that replicate’s parameters, which is brms’ssample_new_levels = "gaussian". That value of the argument is therefore ACCEPTED;"uncertainty"and"old_levels"resample the posterior draws of the levels that were seen and are refused by name. Measured out of sample at unseen levels, the interval used to cover 0.8618 against a nominal 0.95 and itsEst.Errorwas bit-identical to a known level’s. Each DISTINCT unseen level draws its own effect, and rows in the same unseen level share one, which is what makessummary = FALSEright for a mean, a contrast or a sum over new groups.On a MULTIVARIATE fit
predict()returns brms’snrow x 4 x nresparray with the responses named on the third dimension, andsummary = FALSEthendraws x nrow x nresparray. It used to answer for the FIRST response only, in annrow x 4matrix, with nothing said.resp =is a filter on that array. Every response is drawn at the same parameter draw, and underset_rescor(TRUE)the responses are drawn from their joint multivariate normal law, as brms draws them, so the draws carry the estimated residual correlation.predict(summary = FALSE)names neither the draws nor the rows, which is what brms’spredict(summary = FALSE)andposterior_predict()return (list(NULL, NULL), and the responses on the third margin of a multivariate fit).predict(ndraws = 2.5)is an error. The count check ran afteras.integer(), so a fraction was accepted and silently truncated.A simulated parameter draw that makes a distributional parameter non-finite for a ROW makes that one cell NA, instead of failing the whole call in the family simulator; the warning names the rows and how many replicates each lost, and each row is summarized over the replicates it has. No row is summarized over a subset of parameter draws that another row’s overflow selected: the parameter draws and one seed per replicate are taken up front, and a masked cell takes nothing from its neighbours. The rows of one replicate still share one random stream, so a row’s Monte Carlo draws shift with the rows simulated before it. It happens where a parameter the draw law covers is barely identified:
vcov(object, full = TRUE)has its variance, andparam_uncertainty = FALSEsimulates at the estimates alone. Every cell of a response non-finite is an error. -
BREAKING:
fitted(),residuals(),fixef(),vcov(),ngrps()andsummary()return brms’s shapes (item 2.6f).-
fitted()andresiduals()return annby 4 matrix with the columnsEstimate,Est.Error,Q2.5andQ97.5, where they returned a vector.Est.Erroris the delta-method standard error and theQcolumns are the Wald interval atprobs, because there are no draws to take a quantile of.fitted()on an ordinal or categorical fit returns brms’snby 4 byKarray, the third dimension namedP(Y = k); the category probabilities carry a standard error for the first time, by the finite-difference delta method over the whole outer parameter vector. The old value is theEstimatecolumn, andfrm_linpred(fit, type = "response")is the same numbers as a plain vector. -
fitted()now takesprobs,nlpar(a synonym fordpar) andallow_new_levels;ndraws,draw_ids,sort,summaryandrobustare still refused by name. -
residuals()accepts brms’s"ordinary"as a spelling oftype = "response", and takesprobs. Itstypestays in the second position, which is NOT brms’s (brms hasnewdatathere). -
fixef()returns brms’s summary matrix, one row per population-level coefficient under brms’s own name (Intercept,sigma_Intercept,Trt1) and in brms’s order, which puts every predictor’s intercept first. It takes brms’sparsandprobs.fixef(flatten = TRUE)is unchanged and is the vector of estimates inconfint()’s spelling. The per-dpar listfixef()used to return isfixef_by_dpar(), a new export. -
fixef(),vcov()andsummary()$fixedcarry an ordinal fit’s THRESHOLDS and itscs()coefficients, which they left out entirely. Onbf(ord ~ x) + cumulative()they are the rowsIntercept[1],Intercept[2],xand a 3 by 3 covariance, which is what brms gives on the same model; the lane gavexand 1 by 1, andprint(fit)showed a one-row coefficient block. The thresholdEst.Erroris the delta method over the joint covariance, because the family may estimate increments rather than the thresholds themselves. Acs(z)term addsz[1],z[2]after thebblock, as in brms. -
vcov()covers exactly the rowsfixef()reports and names them the same way. It used to be one row wider: a distributional parameter nobody wrote a formula for, such assigmaornu, is an intercept-only linear predictor here and is not a population-level coefficient in brms. It is insummary()’sFurther Distributional Parametersblock instead, on its own natural scale.vcov()now takes brms’scorrelationandpars, andfull,clusterandtypemoved after..., so they must be named.vcov(full = TRUE)is unchanged and keeps the INTERNAL names, because it is the matrix a delta method onconfint()’s rows needs. -
ngrps()returns brms’s named LIST, andNULLfor a fit with no grouping factor. A smooth,gp(),hsgp(),car()orspde()block is no longer counted: those are random-effect blocks here and are not grouping factors in brms. -
summary()carries brms’s$fixed,$random,$spec_parsand$cor_parsslots beside the ones it had, andprint()uses brms’s section headings (Multilevel Hyperparameters:,Regression Coefficients:,Further Distributional Parameters:).$fixedcarries brms’s four columns first and then the Wald test this package reports, where brms writesRhatand the twoESScolumns.summary()now takes brms’spriorsandprob, androbustandmc_seare refused by name;vcovmoved after...and must be named. -
summary()’s EMPTY slots take brms’s empty shapes:$randomisNULLwhere it waslist(), and$spec_parsand$cor_parsare frames with no rows where they wereNULL. -
fitted(),residuals()andpredict()leave their row dimnamesNULL, as brms does. They carried the data’s row names, and underna.action = na.omitthose names were the way to line a result up with the rows of the data that survived; that way is GONE, as it is in brms.frm_linpred()andmodel.frame()still carry them, and the row count of each summary matrix follows the surviving rows. - An ordinal
fitted()cannot report a probability outside[0, 1]. TheQcolumns are a Wald interval, which is unbounded; 15 of 450 bounds were below 0 and 5 above 1 on a three-category fit. They are held inside the range now, so a clamped bound is a sign that the normal approximation is poor there rather than a probability of -0.0055. -
print(fit)prints the summary, as brms’s does.
-
coef()on a mixed ORDINAL fit moves the thresholds by each group’s random intercept, as brms’s does. Forcumulative()andsratio()the group’sIntercept[k]is the threshold minus the group’s mode, and forcratio()andacat()it is the mode minus the threshold, brms’s two sign conventions. brms’s owncoef()on a cumulative and an acat fit reproduces exactly those formulas, draw by draw (dev/shapes-p2-brmsref.R). It used to repeat the thresholds unchanged in every group beside a stray(Intercept)column holding the mode alone.BREAKING:
ranef()names its columns as brms does,Interceptandsigma_Intercept, which is whatcoef()and brms’sranef()say. Code readingranef(fit)$g[, "(Intercept)"]must read"Intercept".BREAKING:
coef()names its coefficients as brms names them,Interceptand not(Intercept), which is whatfixef()andvcov()now say.lmtest::coeftest()keeps only the rows bothcoef()andvcov()name, so the split dropped the intercept row out of a printed significance table with no warning. An ordinal fit’s thresholds arecoef()entries too, for the same reason.fixef_by_dpar()is the design-column spelling.BREAKING:
hypothesis()no longer returns an object of classbrmshypothesis. The class is"frmtmb_hypothesis"alone. The SHAPE is unchanged and so is every element; what stops working isis(x, "brmshypothesis")andinherits(x, "brmshypothesis")in a ported script. frmtmb ownsprint()andplot()for its own class, so neither changes, and a frmtmb object carrying a brms class name was the one thing the class rule forbids.variables()lists an ordinal fit’s thresholds and itscs()coefficients, which it dropped: on asratio()fit with acs()term it listed 3 of 9 parameters, on every ordinal family. They are brms’sb_Intercept[k]andbcs_<term>[k], andhypothesis()reaches them too. The thresholds are reported on the model’s own scale, not the internal (first threshold, log increments) parameterization thatcumulative()andsratio()estimate; the family declares the map inpost$ord_thresholds.insight::get_predicted()has a method of its own. Its default callspredict(x, type = ), whichpredict()no longer has, and it WARNS and returnsNULLrather than erroring, so every prediction was silently lost. The method also carries standard errors, which the default never had:get_predicted(fit)used to reportNAin every CI column.predict = "prediction"ispredict()’s predictive interval and"link"the linear predictor.insight::get_residuals()has a method of its own, insight’sbrmsfitbody.residuals()is brms’snby 4 matrix now, and insight’s default returns whateverresiduals()gives, soget_residuals(fit)began returning 600 numbers where its contract is one per observation. It is theEstimatecolumn again, with the whole matrix on a"full"attribute and insight’s owninsight_residualsclass, which the default never set either.-
marginaleffects::get_coef(),set_coef()andget_vcov(), andinsight::get_parameters()andget_varcov(), read every ESTIMATED coefficient rather thanvcov()’s brms block, because each is paired with a coefficient vector that has the extra rows. Without this the standard errors ofavg_slopes()moved and some becameNA.They also carry an ordinal fit’s THRESHOLDS and
cs()coefficients: on their internal scale underconfint()’s names (tau_raw_1) for marginaleffects, which perturbs and writes the vector back, and on the model’s own scale underfixef()’s names (Intercept[1]), with the delta-method covariance, for insight, which only reads it. marginaleffects builds its Jacobian by perturbingget_coef()one entry at a time, so a parameter the vector left out contributed nothing:avg_slopes()onbf(ord ~ x) + cumulative()reported 0.00029 for the middle category whereMASS::polrreports 0.02016, a factor of 69, on a point estimate that agreed to five decimals. All three categories now agree withpolrto five decimals. This defect predates the shapes work. sim_context()takesmax_iter, the rejection limit atrunc()ed draw needs, whichpredict()exposes as brms’sntrys.No dependency floor moves for any of this.
insight,marginaleffects,emmeansandMASSare all in Suggests already and none of the methods added here needs a newer one:insight::get_residualsandinsight::get_predictedare generics insight has had since 0.9, andmarginaleffects’s four extension generics are unchanged.lmtestis used bydev/shapes-interop.Rand by nothing in the package.
frmtmb 0.60.0
tmbstan (>= 1.2.1)in Suggests. tmbstan 1.2.1 is the first build that samples the model rather than a standard normal when built against StanHeaders 2.39 (dev/tmbstan121-findings.md).-
BREAKING: every condition frmtmb raises is classed, the contract brms keeps with
brms_error. An error has the classc("frmtmb_error", "error", "condition"), a warningc("frmtmb_warning", "warning", "condition")and a messagec("frmtmb_message", "message", "condition"), sotryCatch(frmtmb_error = )catches any refusal. A condition that an extension raises also has a subclass named for the package, such asfrmtmb_eam_error. See?frmtmb-conditions.What changes for a caller: the class vector no longer contains
simpleError,simpleWarningorsimpleMessage, so a handler or test that catches those classes by name stops matching. A handler forerror,warningormessageis not affected. Afrmtmb_fit_error, raised when the optimizer fails, is now afrmtmb_errorand is no longer asimpleError. The one exception is the refusal ofmodel.matrix()on a multivariate fit, which isc("frmtmb_error", "simpleError", "error", "condition"): insight 1.5.4 tests forsimpleErrorto giveNAstandard errors inget_predicted_ci(), and without it that call stopped with “argument is of length zero”.Unlike brms’s class vector, frmtmb’s does not contain
rlang_error: frmtmb builds the conditions with base R and does not import rlang. Errors from other packages, such as RTMB, TMB and Matrix, keep their own class, and so does R’s “unused argument” for an argument that a function does not have. -
BREAKING: refusals that base R used to raise now have frmtmb’s class and new text. Each is a
frmtmb_errorwhose message names the argument or variable, the value given and what is allowed. Code that matches the old text stops matching:- A value that matches none of the choices of
predict(type =),fitted(scale =),residuals(type =),confint(method =),drop1(test =),hypothesis(method =, scope =),conditional_effects(band =, method =),frmtmb_control(check_nlev_1 =, check_olre =),default_prior(route =),get_prior(route =),validate_prior(route =),vcov_cluster(type =),frm_periodogram(taper =, detrend =)andanova()on afrmtmb_multiple(method =,use =). The old text was'arg' should be one of "response", "linear". The new text is`scale` must be one of "response", "linear", not character "latent". A partial value such as"resp"still matches. - A prior string with the wrong number of arguments, for each density that
set_prior()reads. The old text waslength(pars) == 2 is not TRUE. The new text isPrior 'normal(0)' gives 1 argument; normal() takes 2: normal(mu, sigma). Anexponential()rate that is not positive, and a prior that is not one string, have messages of their own. - An argument of the wrong type that a
stopifnot()refused with the text of the assertion:set_prior(...) + "x",frm(prior = list(1)),diagnose(),cluster_scores()andvcov_cluster()on an object that is not a frmtmb fit,check_custom_family()on an object that is not a family, andfrmtmb_family()with afamily,dparsorlpdfof the wrong type. -
predict()and the other methods that takenewdatarefuse anewdatathat lacks a variable of the model (wasobject 'z' not found, nowVariable 'z' missing from newdata) or that has a level of a population-level factor that the fit did not see (wasfactor f has new levels zz). brms refuses both. -
frm()refuses a variable that is neither a column ofdatanor an object found from the formula (wasobject 'nope' not found), adatathat is not a data frame, a list column used as a variable (wasinvalid type (list) for variable 'lc'), and an object from the formula environment whose length is not the number of rows (wasvariable lengths differ). -
hypothesis()refuses a name in backticks, such as`(Intercept)`, as it refuses other unknown parameters (wasobject '(Intercept)' not found). -
lme4::getME(<multivariate fit>, "Zt")keeps its class. The text loses the prefixerror in evaluating the argument 'x' in selecting a method for function 't':that R’s S4 dispatch added.
- A value that matches none of the choices of
frm_stop(),frm_warning()andfrm_message()are exported for extension authors. They take the arguments ofstop(),warning()andmessage()and add the classes. Given one condition object, they signal that object with the classes added, as the base functions do.frm_warning(immediate. = TRUE)prints at once whengetOption("warn")is 0. A helper given as a value, as inlapply(x, frm_stop), gets the subclass of the function that gave it.package =names the package whose subclass the condition gets.frm_match_arg()ismatch.arg()with a classed refusal, andfrm_family_package()gives the package that built a family.A refusal that frmtmb raises from text an extension wrote as data, such as
frmtmb_structure(refusals =), or about a declaration of a family that an extension built, such as an addition term the family needs, has the subclass of that extension. For example,frm(..., family = hmm(...), REML = TRUE)is afrmtmb_latent_error.?frmtmb-conditionssays which refusals have the subclass and which do not.Every extension in this repository now requires these functions, so the
frmtmb (>= 0.59.0)floor of frmtmb.coupling, frmtmb.eam, frmtmb.latent, frmtmb.learn, frmtmb.ode, frmtmb.sample and frmtmb.spline must move to the release that includes this change.The worked-example families in
inst/bcm/andinst/rl/, whichvignette("bayesian-cognitive-modeling")andvignette("reinforcement-learning")load withsource(), raise their refusals withfrm_stop()too, so a copy of them keeps the classes.
frmtmb 0.59.0
BREAKING, and the silent wrong answer it removes. The link for the mean is now checked against the set brms 2.23.0 allows for the family, and a link outside it is refused with brms’s sentence and the set:
'sqrt' is not a supported link for family 'bernoulli'. Before, onlylink_<dpar>was checked, sobernoulli("sqrt")FITTED, returning coefficients for a probability oneta^2, which is not bounded by one. What stops working: every (family, link) pair brms refuses, 417 pairs over 37 families, of which 371 used to construct. Among them areexponential("cloglog"),Beta("1/mu^2"),zero_inflated_negbinomial("logit"),beta_binomial("log"), astatsfamily carrying such a link (frm(..., family = poisson("inverse"))), and any link but"logit"oncategorical()reached by name. Every pair brms accepts still constructs and fits: 207 fits give the same log-likelihood as before to ten decimals. The sets are read out of brms bydev/famlink-gen-links.R, not typed.nbinom1(),tweedie()andhuber()have no brms family and take the set ofnegbinomial,Gammaandgaussian. A custom link object is not held to a set.A family object carries brms’s fields.
$linkis the name of the link for the mean,$linkfunand$linkinvare its functions, and$link_<dpar>names the link of every other parameter:beta_binomial()$link_phiis"log".$linkused to partial-match thelinkslist and return it. The fields are not stored:$computes them fromlinkswhen they are read, so they describe the links a fit uses however the object was edited, and a fit saved by an earlier version answers them too.names()and[[do not show them. BREAKING:$on a family object no longer partial-matches. A name that is only a prefix of one field (fam$lpd) is an error naming the field. An element stored on the list under one of those names is an error when$reads it.BREAKING:
print()on a family object prints brms’s first two lines,Family:andLink function:, and then the parameters with their links. The<frmtmb family>header is gone.The link for the mean may be unquoted, as
stats::family()and brms allow:student(identity),negbinomial(sqrt),zero_inflated_poisson(log). A bare link name is read as the name without being evaluated, and a variable holding a link name is read for its value, which is the order brms uses.link = NULLandlink = NAgive the default. A bare link name the family does not take and that is bound to nothing, as innegbinomial(inverse), is refused as that link, where brms reports “object ‘inverse’ not found”.New
brmsfamily(), brms’s generic constructor:brmsfamily("gaussian", inverse),brmsfamily("zi_poisson"). A family name now follows brms’s spelling rules wherever a name is accepted: case does not matter,"normal"is"gaussian","com_poisson"is"compois", and"zi_"and"hu_"stand for"zero_inflated_"and"hurdle_". An unknown name is refused with brms’s sentence,x is not a supported family.BREAKING:
frm_family()is removed.brmsfamily()is the one spelling, and takes the same arguments with the link second.frm(family =)takes the family and its link as one vector,c("weibull", "log"), which brms accepts forfamily =; a vector of three or more is refused.Changes fits, and the silent wrong answer it removes. A brms family object passed to
frm()now keeps itslink_<dpar>links. They used to be dropped without a word, sofamily = brms::student(link_sigma = "identity")fitted sigma on the log link the user had not asked for. Such a model now fits the link it names, and its estimates move.A correct fit no longer warns “NA/NaN function evaluation” when nlminb’s line search steps past the domain of a link, such as below zero on
inverse.gaussian’s1/mu^2orGamma’sinverse, or outside the unit interval onbernoulli("identity"). Such a trial value is handed to nlminb as+Infrather thanNaN; nlminb backtracks from both identically, so every optimum is unchanged (212 brms link pairs and five inverse gaussian designs giveidentical()estimates). A model whose objective isNaNat its starting values still warns. The number of mapped trials, summed over every optimizer run of the fit including restarts, isfit$opt$nonfinite_trials. When it is not zero,diagnose()prints it and returns it asnonfinite_trials, andverbose = TRUEadds it to the optimizer stage lines and to the finaldoneline.BREAKING, changes fits:
inverse.gaussiantakes brms’s default link,1/mu^2, where frmtmb’s waslog. It reaches every fit that names the family as a string or throughbrmsfamily()without a link, and those estimates move.stats::inverse.gaussian()already carried1/mu^2and is unaffected. Writebrmsfamily("inverse.gaussian", "log")for the old model. Fits move, and some designs converge poorly on this link: over 12 designs of 20 data sets each, 135 of 240 default-link fits ended with code 0 and no warning, against 234 of 240 onlog. The failures are warnings such as a large gradient or a Hessian that is not positive definite, never a silent wrong answer. If a fit warns, passlink = "log".BREAKING:
binomial(),beta_binomial(),zero_inflated_binomial()andmultinomial()without atrials()term are refused with brms’s message,Specifying 'trials' is required for this model., and so is amixture()with such a component.multinomial()used to take its trials from the row sums of the response; itstrials()must now equal them, as in brms. The others used to be fitted as one trial per row. Writey | trials(n) ~ ..., orbernoulli()for a 0/1 response.cbind(successes, failures)supplies the trials and is unaffected.get_prior()does not refuse these responses, because brms’sget_prior()does not and no prior depends on the trials.BREAKING:
mixture()refuses what brms refuses. Components with real and integer support (mixture(lognormal, exgaussian, poisson())) are refused, because a weighted sum of a density and a probability mass is not a likelihood.categorical(),multinomial(),hurdle_gamma(),hurdle_lognormal()andzero_inflated_beta(), the families brms’sno_mixture()bars, are refused as components, and a mix of ordinal and non-ordinal components is refused with brms’s sentence. A named argument in the dots, such as brms’snmixororder, is refused by name; it used to be read as a component and reported as an unsupported family.BREAKING: an unordered factor response to
cumulative(),sratio(),cratio()oracat()is an error, as in brms. It used to warn and fit with the alphabetical level order as the category order. Use an ordered factor or integer codes. The ordinal response refusals open with brms’s sentence,Family 'cratio' requires either positive integers or ordered factors as responses.frm()messages brms’s suggestion,Only 2 levels detected so that family 'bernoulli' might be a more efficient choice., for abinomial(),beta_binomial()orzero_inflated_binomial()response whose trials are all one, including amixture()of those, and for an ordinal or categorical response with two categories.The
link_<dpar>refusal now opens with brms’s sentence,'logit' is not a supported link for parameter 'shape'.BREAKING. A group-level coefficient that two terms of one predictor both give to the same grouping factor is refused, as brms refuses it.
y ~ x + (1 | g) + (x | g)used to fit two independent intercept blocks ong, and only the sum of their variances is identified: on a 240-row construction the two intercept standard deviations came out 0.56151 and 0.56093, the log likelihood equal to that of(x | g)alone (dev/priorform-base-behaviour.R). What stops working: every such repeat, including(1 | g) + (x || g),(1 | g:h) + (1 | g/h)and(1 | mm(g, h)) + (x | mm(g, h)). Write the coefficient in one term. This includes the animal model’s(1 | gr(id, cov = A)) + (1 | id), which IS identified through the matrix but which brms also refuses; write the permanent-environment term on a copy of the column,(1 | id2), as brms does, and the message says so. The glmmTMB structures brms does not have follow the same rule:rr(...) + diag(...),equalto(...) + us(...)andar1(...) + diag(...)repeating a coefficient on one factor are refused.A bar term written twice,
(1 | g) + (1 | g), is one block, as brms reads it; it used to fit two blocks splitting one variance at will. Grouping factors are compared as brms writes them, so(1 | g:h) + (1 | h:g)is two blocks, as in brms, while(1 | g:h) + (1 | g/h)repeats one and is refused. A multi-membership group is named by its members, as brms names it, so(1 | mm(g1, g2)) + (1 | mm(g1, g2, weights = cbind(w1, w2)))repeats the intercept and is refused; it used to fit two blocks. A whole-term special written twice,s(z) + s(z), is one term, as in brms; it used to build two smooths with one column name. A split that repeats no coefficient, such as(1 | g) + (0 + x | g), and the same term in two predictors, such asbf(y ~ (1 | g), sigma ~ (1 | g)), are still accepted. On 489 generated designs and 158 formulas harvested from brms’s own tests, vignettes and examples, the refusal fired on none that brms accepts and missed none that brms refuses (dev/priorform-falsealarm.R).BREAKING, and the defect it fixes.
cs(), a smooth (s(),t2(),te(),ti()),gp(), an autocorrelation term andmmc()inside an interaction or another call are refused by name, which is brms’s list of whole-term specials.y ~ x * cs(g)used to fity ~ x, and so didy ~ x:s(z)andy ~ x * s(z);y ~ I(s(z))fitted an intercept alone. The dropped terms went without a message. The message names the term as brms does,The term 'x:s(z)' is invalid, and points a smooth atby =.s(z) * s(z), whichterms()collapses tos(z), is read as that one term, as brms reads it.BREAKING. A formula whose right-hand side is itself a formula,
y ~ ~ x, is refused by name inbf(),lf(),nlf()and in the parameter formulas ofbf(), as brms refuses it (brms issue #749). Before,frm(y ~ ~ x)fittedy ~ x.bf()returns a formula it already built unchanged, soidentical(form, bf(form))holds as it does in brms. It used to stop with “formulamust be a formula”. Further arguments add to that formula; a parameter it already sets is refused, as+ lf()refuses it.nlnow defaults toNULL, as in brms:FALSEfor a new formula, and the existing setting for a built one.bf(mvbind(y1, y2) ~ ..., nl = TRUE)now passesnlto each response, where it used to drop it.bf()andlf()take a one-sided parameter formula named by its parameter,bf(y ~ x, sigma = ~ z), which is brms’s second spelling ofsigma ~ z. A one-sided formula with no name is refused with brms’s wording, “Additional formulas must be named”, and an invalid parameter name now says it “must not contain dots or underscores”.BREAKING. Two prior specifications for the same slot (class, coef, group, resp, dpar and nlpar) are refused by
frm(),frm_sample(),frm_simulate(),par_template()andvalidate_prior(), as brms refuses them. What stops working: a repeated specification, two densities for one slot, and a density followed by a bounds-only specification, which used to tighten it. Write the density and both bounds in one call. Slots are compared field by field, soclass = "b", dpar = "sigma"andclass = "b", coef = "sigma"are two slots although brms prints both asb_sigma. Onfrm_sample()of a fit that carries its own prior, a specification on the call for a slot the stored prior names replaces the stored one, as brms’supdate(prior = )replaces a row, and is not refused.BREAKING. When specifications for different slots reach one parameter, the more specific one applies whatever the order, as in brms: a
coefover its class, and agroupover a class-widesdorcor. The later one used to win, so a coefficient prior written BEFORE its class prior was silently replaced by it. Fits change only for that order: on the construction indev/priorform-punch-order.R(seed 20260916),coef = "x"then classbmoved from a log likelihood of -261.34 to -409.94, the value the other order always gave, and a groupsdprior then a classsdprior on a beta model from 211.86 to 215.28. The same-order fits are bitwise unchanged. Between different classes,corand thethetahatch, the later one still applies.set_prior()takes vectors in every argument and returns one specification per element, recycled as a data frame column is:set_prior("normal(0, 2)", class = c("b", "sd"))is two specifications, as in brms. A length that does not divide the longest is refused.BREAKING. A prior object prints in brms’s layout. One specification prints as
b_x ~ normal(0, 1), with bounds in front as<lower=0>; several print as a table with brms’s columns, andprint(x, show_df = FALSE)gives one line each; a row with no density of its own shows its class row’s as(vectorized), and an empty prior prints nothing, as in brms. What stops working: code that reads the oldnormal(0, 1) class=b coef=xline. The density prints as it was written, soset_prior("cauchy(0,1)")printscauchy(0,1)rather than thestudent_t(1, 0, 1)it parses into.A prior object answers brms’s column reads:
pr$prior,pr$class,pr$coefand the rest return one element per specification, from the tableas.data.frame()now builds. Assigning a column,pr$prior[2] <- "exponential(1)", rebuilds the specifications withset_prior().default_prior()is new and is the canonical name, as it has been in brms since 2.20.14. BREAKING:get_prior()is now its alias,get_prior(formula, ...), which is brms’s signature. Calls that passdata,family,data2orrouteby position or by name are unchanged.BREAKING. The
default_prior()table has brms’s columns in brms’s order,prior, class, coef, group, resp, dpar, nlpar, lb, ub, source;respused to come afternlpar, andsourceis new. A class"sd"or"cor"row for a block of a distributional parameter’s predictor now names that parameter indpar. Before,bf(y ~ (1 | g), phi ~ (1 | g))listed onesdrow forgwhere the model has two blocks.validate_prior()is new. It resolves a prior against a model with the resolutionfrm()uses, refuses it with the same message when any part of it addresses nothing, and returns the wholedefault_prior()table with the prior filled in andsourcemarking the rows set and the rows that inherit a class row’s density.frm(prior = )accepts that table, and adefault_prior()table edited in place, and fits the same model as the prior that built it.empty_prior()andas.brmsprior()are new.as.brmsprior()turns a data frame with apriorcolumn into a prior object, fillingset_prior()’s defaults and dropping other columns, as brms’s does; here the result is thefrmtmb_priorlistthatset_prior()returns.-
BREAKING:
variables()andhypothesis()use brms’s names. Every coefficient carries brms’sb_prefix, and a coefficient of a distributional, nonlinear or multivariate predictor is spelled the way brms spells it:b_Intercept,b_x,b_sigma_Intercept,b_sigma_y1_x,b_a_Intercept.variables(fit)used to returnIntercept, zBase, ..., sd_patient__Intercept, half of brms’s convention. A group-level standard deviation of a distributional parameter issd_g__sigma_Intercept; it used to be a secondsd_g__Intercept, which the mean’s block claimed, so it could not be reached at all. An|ID|block’s names aresd_id__y1_Intercept, notsd_id__y1.muIntercept.Every piece of a name goes through one renamer ported from brms’s
rename()and its helpers:I(x^2)isb_IxE2, a factor levelc-disb_fcMd, a responsey_aisya(b_ya_x,sigma_ya), an interaction group keeps its:(sd_g:h__Intercept) and joins its levels with_(r_g:h[1_p,Intercept]), whitespace in a level is a dot in anr_name, and a smooth’s unpenalized column isbs_sx_1with smoothing SDsds_sx_1. A distributional parameter nobody wrote a formula for is the parameter itself on its natural scale,sigma,shapeorsigma_ya, and has nob_sigma_Intercept; written assigma ~ 1it isb_sigma_Interceptand there is nosigma, as in brms. The autocorrelation parameters arear[1],ma[1],cosy, and a multivariate model’s residual correlationsrescor__ya__yb. Checked against the names brms derives for thirteen models, including those hostile names and the unmodeled shape, nu, phi, zi and hu: 210 of 210 present and none extra, 12 of 210 before (dev/brmsnames-findings.md).What stops working: a bare coefficient name in
variables()’s output ("x" %in% variables(fit)), ahypothesis()string that names a natural-scale quantity withoutclass = NULL,b_sigma_Intercepton a model whose sigma has no formula (usesigma), andar1(usear[1]). BREAKING:
hypothesis()takes brms’sclass = "b"default and brms’s argument order,(x, hypothesis, class, group, scope, alpha, robust, seed, ...), withmethod,nsimandvcovafter them. A bare name is read as the coefficient,xasb_x, as in brms, sohypothesis(fit, "x1 - x2 = 0")still works. A natural-scale name needsclass = NULL, as in brms:hypothesis(fit, "sd_g__Intercept / sigma = 0", class = NULL). The string is read asbrms:::eval_hypothesis()reads it: every variable gets theclassandgroupprefix, a prefixed name the model does not have is refused with brms’s “Some parameters cannot be found in the model”, and brms’s renaming (:to___,[and]to.,,to..) is applied before parsing.x:feis therefore the interaction coefficient. It used to parse as R’s:operator, which returnedb_xwheneverb_xandb_x:fewere less than 1 apart:hypothesis(fit, "x:fe > 0")gave 0.2415 for a coefficient of 1.0794, with no error. A name written in full under the default class ("b_x = 0") is refused, as brms refuses it, and the backquoted internal spelling`(Intercept)`is no longer an alias.alphaused to be the third positional argument, sohypothesis(fit, h, 0.1)now names a class.robust = TRUEandscope = "ranef"or"coef"are refused by name: both need draws.BREAKING:
hypothesis()returns brms’s object. A list of classc("frmtmb_hypothesis", "brmshypothesis")with brms’s elementshypothesis,samples,prior_samples,classandalpha, whose$hypothesisframe has brms’s eight columns:Hypothesis,Estimate,Est.Error,CI.Lower,CI.Upper,Evid.Ratio,Post.Prob,Star. On a fit,Est.Erroris the delta-method (or bootstrap, or pooled) standard error, the interval follows brms’s rule (central1 - alphafor=, central1 - 2 * alphafor a directional row, so its relevant end is the one-sided bound; it used to be infinite on the other side),Evid.RatioandPost.ProbareNA, and the test statistic and p-value areattr(h, "test"). What stops working:h$estimate,h$se,h$lwr,h$upr,h$z,h$p,nrow(h), andas.data.frame(h).BREAKING: the reserved-name note and the dot spellings are gone. A covariate named
sigmaused to shadow the residual SD inhypothesis(), with a message and a.sigmaescape. With brms’sb_prefix the coefficient isb_sigmaandsigmais the residual SD, so that pair no longer meets.hyp_shadow_arm()andhyp_shadow_disarm()are no longer exported.-
BREAKING: names that brms’s renaming would give twice are handled as brms handles each case. Measured on brms 2.23.0 (
dev/brmsnames-rev-collide-brms.R):- two design columns of one predictor that rename alike,
y ~ Intercept + x, are refused when the model is built, with brms’s “Internal renaming led to duplicated names”; - a name given by two predictors takes brms’s
__1suffix on the later one:bf(y ~ sigma_z, sigma ~ z)hasb_sigma_zandb_sigma_z__1, where the second used to be a duplicate thathypothesis()could not reach; - the same group-level coefficient on one group twice is refused with brms’s “Duplicated group-level effects are not allowed”. The animal model
(1 | gr(id, cov = A)) + (1 | id)is refused this way; give the second term a copy of the column,(1 | id_pe); - two responses brms spells alike,
y_aandya, are refused with brms’s “Cannot use the same response variable twice”; - a group-level label given twice, the
r_gd[lvl.1,Intercept]of the levelslvl 1andlvl.1, takes brms’s__1on the later level; - an interaction group two of whose levels brms joins to one string,
1_2:3and1:2_3in(1 | gi:hi), is refused. brms fits them as one level and pools their effects; recode the factors.
No name reaches
variables(),hypothesis()or the draws labels twice. What stops working: the models above that are refused. - two design columns of one predictor that rename alike,
BREAKING: a mixture’s weights are brms’s
theta1 ... thetaK. With no theta formula written,variables()andhypothesis()name the mixing PROBABILITIES, which sum to one, as brms does. They used to name the estimated log ratiotheta1(1.966 where the share was 0.876), sohypothesis(fit, "theta1 = 0.5", class = NULL)gave 1.466 where brms gives 0.376.frm_simulate(newparams =)takes the whole simplex,theta1 = 0.9, theta2 = 0.1, and refuses a part of it; it used to readtheta1 = 0.7as a log ratio, a share of 0.665. A mixture with a theta formula keeps the log ratios as coefficients,b_theta1_Intercept. Anhmm()transition logit is likewise the coefficientb_tr12_Intercept, nottr12.BREAKING: two more names are brms’s. A monotonic term’s scale is
bsp_moxo, as brms names the same quantity (it wasb_moxo), and a by-smooth’s smoothing SD issds_sxf2u_1(it wassds_sx:f2u_1).BREAKING:
VarCorr()returns brms’s structure. A list keyed by GROUPING FACTOR ("patient", as brms and lme4 key it, not"1 | patient"), each entry withsd(coefficients x statistics) and, for a group with correlations,corandcov(coefficient x statistic x coefficient arrays), thenresidual__where brms has it: asigmawith no formula, one row per response on a multivariate model (namedya,y2) with the residual correlations underrescor = TRUE, and noresidual__once any response predicts sigma. The statistics areEstimate, its delta-method standard error asEst.Error, and Wald quantiles atprobs. Several blocks on one factor merge under it, as in brms. Smooth, GP, CAR and SPDE blocks are not in it, as in brms;confint_varcorr()reports them. A model with no group and no scalar residual SD is refused with brms’s message. What stops working:VarCorr(fit)[[1]]as a covariance matrix,names(VarCorr(fit))as term labels,as.data.frame(VarCorr(fit)), andprint()’s lme4 table fromVarCorr()(the fit’s ownprint()andsummary()keep it). The per-block matrices arevarcorr_matrices(fit)on?frmtmb-sampling-api.BREAKING:
fixef(),ranef(),coef()andVarCorr()take brms’s leading arguments in brms’s positions (summary,robust,probs, andparsandgroupswhere brms has them).fixef(fit, FALSE)andranef(fit, FALSE)used to setflattenandcondVarand return the default shape,identical()to the call without the argument. A fit has no draws, sosummary = FALSEandrobust = TRUEare refused by name with the reason; brms’s defaults spelled out are accepted.flattenandcondVarmove after...and must be named.posterior_summary()on a fit, or on afrm_multiple()result, refuses and says to sample first. It used to die insideposterior_summary.default()with “is.atomic(x) is not TRUE”.posterior_summary.default()is brms’s, statistic for statistic, and now summarizes a three-dimensional draws array too.confint(parm =)andprofile(parm =)take brms’sb_spelling of a coefficient as well.BREAKING:
frm_simulate(newparams =)takes brms’s names only, the stringsvariables()returns:b_Intercept,b_x,b_sigma_Intercept,sigma,sd_g__Intercept,sds_sx_1. A bare name (Intercept,x,sigma_Intercept) is refused, and the message gives the brms spelling of each. The prior-draw tableattr(sims, "pars")uses the same names. Priors do not change:set_prior(coef = "x")takes the bare coefficient, as in brms. What stops working: every natural-scalenewparamslist written with bare names; the internal spelling (beta,betad,theta) is unchanged.
frmtmb 0.58.0
-
BREAKING, and the rule behind it. Where lme4 or glmmTMB and brms disagree on an argument name, brms is now the tiebreaker: this package takes brms’s spelling and DROPS the other one. Two spellings of one setting survive only where brms itself carries both.
predict(),simulate(),frm_bootstrap()anddharma_residuals()now takere_formula. lme4’sre.formis gone from all four and is refused by name, with the replacement in the message.predict()no longer acceptsallow.new.levelseither;allow_new_levelsis brms’s spelling and the only one.pp_check()on a fit dropsre.formtoo, because brms’s ownpp_check()forwards toprepare_predictions(), whose formals carryre_formulaalone. -
BREAKING, and the defect it fixes.
fitted()took only..., so every brms argument handed to it was SWALLOWED.fitted(fit, re_formula = NA)returned the conditional fit and said nothing: 1.291651 at the first row of the fixture intests/testthat/test-arg-refusal.R, bit-identical tofitted(fit), where the population answer asked for is 1.954262 and the largest gap over the 120 rows is 3.065438. A misspelled argument was swallowed the same way.fitted()now takesnewdata,re_formula,scale,respanddpar, which isfitted.brmsfit()’s signature less the arguments that need draws, and it is implemented as one call topredict(), so the documentedfitted() == predict(type = "response")identity is structural rather than two bodies that agreed. -
BREAKING, and the reason it is worth the break. An argument that lands in a method’s
...is now an ERROR naming it. 145 registered S3 methods across this package andfrmtmb.sampletook a...they never read; twelve of them keep it, and the rest refuse. A method that swallows its dots accepts every argument brms adds in future, and every misspelling today, in the same silence.predict(),conditional_effects()andhypothesis()warned rather than erroring; a warning still returns a number, and undersuppressWarnings()or in a loop it leaves no record.tests/testthat/test-arg-refusal.Rasserts the invariant structurally: no registered S3 method may have a...it never touches. Two shapes are outside it. Twelve methods of generics that emmeans, insight and marginaleffects own are called BY those packages with arguments of their own choosing, so accepting an unknown name is the contract there (insight:::.n_parameters_componentcallsget_parameters(x, component = ), which our method has no formal for). And a method whose whole body is an unconditional refusal cannot swallow anything, so it keeps its own message; the test recognises that shape rather than listing those methods by name. An argument brms has that a maximum likelihood fit cannot answer is refused BY NAME with the reason, instead of being reported as an unknown name.
fitted(fit, ndraws = 10)says that this fit carries one estimate rather than a posterior and points atfrmtmb.sample::posterior_epred(). The same holds fordraw_ids,sort,summary,robust,probsandnlparonfitted()andpredict(); fornewdata,re_formula,methodandresponresiduals(), which reads the fitted rows only; fordigitsandshortonprint(); forpriors,probandmc_seonsummary(); forsummary,robust,probsandparsoncoef(),fixef(),ranef()andVarCorr(); forresponnobs()andfamily(); and forcorrelationandparsonvcov().Arguments that R passes through its OWN generics are accepted, not refused, because R’s S3 contract is that a method tolerates what its generic carries.
as.formula(model)isformula(object, env = );model.matrix.default()andaov()callterms(object, data = );step()callsnobs(object, use.fallback = )and thendrop1(object, scale = , trace = ). All of those worked before the dots were refused and broke afterwards. One table,s3_contract_args, now covers every generic at once, and a test walks R’s own packages for the same pattern so the next name of this kind fails the suite rather than reaching a user. Everything outside the table is still refused, sonobs(fit, nosucharg = 1)andterms(fit, re.form = NA)error as before.residuals()forwarded its...toTMB::oneStepPredict()on thetype = "osa"branch and swallowed them on every other one. The dots are now checked against that function’s own formals under"osa"and refused otherwise.
frmtmb 0.57.0
-
frm_install_generics(pkgname, owners)joins the extension API on?frmtmb-sampling-api. It is the mechanism frmtmb uses on itself, andownersis now an argument, so an extension that defines a generic under a name another package owns calls it from its own.onLoad()with its own table.frmtmb.sampledoes this for 28 names. A malformed table is refused rather than installing nothing.
frmtmb 0.56.0
-
frmtmb no longer breaks brms, lme4, posterior, loo, rstantools or bayesplot for objects that already exist. Attaching frmtmb after one of them used to cost that package every S3 method it had on a name the two share, because frmtmb defined and exported its own generic and
UseMethod()reads the method table of the namespace where the generic it reached was defined. Measured on the previous release:library(brms); library(frmtmb)thenloo(fit)on an existingbrmsfitlost all 27 of the generics the two share, two of them SILENTLY by falling into frmtmb’s own.default; it is now 0 of 27, in every load order tested.library(lme4); library(frmtmb)lost all 5 of the names lme4 and frmtmb share; it is now 0 of 5.frmtmb now shares the owner’s generic instead of shadowing it.
fixef(),ranef()andVarCorr()are imported from nlme, which is Recommended and is where lme4, glmmTMB and brms get them too, andrefit()from generics. The rest resolve to the owner’s generic at run time when the owner is loaded, and to frmtmb’s own when it is not, so nothing new appears in Imports. -
BREAKING, one call.
posterior_summary()’s first argument is nowx, notobject, matching brms’s generic, soposterior_summary(object = m)fails andposterior_summary(x = m)orposterior_summary(m)is the spelling. That is the only caller-visible break:nvariables()andVarCorr()change too, but both only GAIN an argument (...from posterior,sigmafrom nlme), so no call that named their first argument breaks.refit()is a fourth changed signature and a subtler one: it is(object, ...)from generics normally and(object, newresp, ...)when lme4 is loaded, because lme4 defines its ownrefitgeneric rather than importing one, soargs(refit)now depends on what else is loaded. No caller breaks on it.If you write an S3 method on any of these names, match the OWNER’s formals, not frmtmb’s.
frmtmb.sampleis updated here for that reason and itsfrmtmb (>= ...)floor moves with this release. as_draws(),as_draws_array(),as_draws_df(),as_draws_list(),as_draws_matrix(),as_draws_rvars(),ndraws(),nchains(),niterations()andnvariables()now refuse afrmtmb_fitby name and point atfrmtmb.sample::frm_sample(). They had no method for the class at all, which was R’s own “no applicable method” before and would have become posterior’s “All list elements must be lists themselves” after, since a fit is a bare list.hyp_shadow_arm()andhyp_shadow_disarm()join the extension API on?frmtmb-sampling-api. Ahypothesis()method in another package must arm the reserved-name shadowing note itself, because the generic that dispatched to it may be brms’s;frmtmb.sample’s method for draws had relied on core’s generic doing it and lost the note in every session until it did.posterior (>= 1.0.0)is declared in Suggests. frmtmb registers methods onas_draws_rvars()andas_draws_list()now, and a partial or pre-releaseposteriorthat lacks one of them stops frmtmb from loading. posterior 1.0.0, its first CRAN release, already exports both.?frmtmb-scalesis new and says, per method, whether a number is on the link scale, the response scale or neither. Two places where frmtmb and brms disagree are written down there rather than left to be discovered:predict()returns the linear predictor where brms returns the response scale, andsummary()printssigmaon its log link wheresigma()back-transforms.
frmtmb 0.55.2
-
?frmtmb-linksdocuments a return value. pkgcheck reports any help page with no\docType, nodatasetsorinternalkeyword and no\valueas a function with no documented return, and this was the only page in the package that matched. Written the way?frmtmb-autocorand?frmtmb-multimembershipalready answer it, since those are concept pages too.
frmtmb 0.55.1
?frm_compat_featuresnow says which column a caller may key on.namewas documented as “how the feature is written in a formula”, which is false for the three_predrows: a formula writesmi(x), nevermi_pred(x).keyis the parser name, and frmtmb.sample’s new pre-flight refusal depends on that, so it is a contract rather than an observed regularity now. No behavior changed.?frmtmb_register_compatstates what arefusedrow promises: the row is a claim about what the code does, not a note for a reader, and it is enforced wherever the enforcing code can see the feature, with that code saying what it can see. Most of core’s refusals are hand-written guards the row describes, and those see everything. The other kind CONSULTS the registry at the point of use, andfrmtmb.sample::frm_sample()is the first caller of that kind, so a package that owns a model the sampler cannot run declares one row and needs no code in the sampler. The qualification is where the two differ: deciding whether a model uses a feature usually means knowing WHERE its call sits in the formula, which the parser owns and nothing public carries out of core, so a consulting caller documents the rows it cannot decide and warns rather than refusing on those. Documentation only; the registry itself is unchanged.
frmtmb 0.55.0
Two densities were silently wrong and are fixed. The student-t log density cancelled its own digits as nu ran off, by 57 log units at nu = 1e50, which the grammar fuzz tier had recorded as two optima under a row permutation; and a Bayesian Cognitive Modeling family read a probability on the wrong scale, by 1.5716 nats per observation on the identity link. Neither is a regression: both have been wrong for longer than 0.54.0.
The accessors that make a density exact at a saturated link are now public, which is what a family in another package needs and had to write again. se() reads the family’s own declaration of which scale the known standard error replaces. whittle() stops refusing tapered periodograms. The hazard-container lint runs inside each extension’s own check rather than only at the release tally.
-
A custom family can now write a numerically exact density from another package.
dpar_log(),dpar_log1m(),dpar_log_complement()anddpar_complement()are exported: they read a distributional parameter on the LINEAR-PREDICTOR scale, which is the only scale that survives a saturated inverse link.stats::plogis(eta)is exactly 1 in double precision frometa = 36.7368005696771up, so a density that forms1 - muby subtraction gets exactly 0 there and returnsNaNfor the value and for the gradient; below that the complement is wrong by 1.0e-3 relative ateta = 30. Both regions are reachable in real fits. The accessors were internal, so a family defined outside frmtmb could only write the arithmetic again:frmtmb.couplingdid exactly that and now calls the public API instead. They take the dpar’s OWN link, as a name or as a link object, because the linear predictor is on the link scale and reading it as a log odds when the link is a probit is a different, wrong density. See?"frmtmb-robust-dpars".Take the one-sided
dpar_log()ordpar_log1m()when the density needs one term.dpar_log_complement()returns both from one log odds, which is what a mixture gate wants, and records a secondRTMB::logspace_add()over the whole response that the tape then replays on every gradient evaluation. At 1000 rows the accessor tapes 14002 AD nodes against 10002; one gradient sweep of thecross_wishart()density costs 633 us against 480 us, and 462 us for the hand-written arithmetic it replaces; the accessor measured on its own is 387 us against 205 us. Six interleaved blocks of 2000 sweeps, one process. The R-level density call is not the cost that matters here: it runs once per fit, when the tape is built.The
.eta_<dpar>entry these read is documented as RESERVED, not as API. It is visible indparsand it is not a supported thing to read: its meaning depends on the dpar’s link, and whether it is present depends on whether the objective is being taped. The accessors fold both branches in, which is the whole reason to have them.check_custom_family()supplies no linear predictors, so it checks the fallback branch only, and now says so. -
bcm_contaminant()ininst/bcm/binomial-extras.Rfitted a different density from the one it documented wheneverlink_phiwas not the default. It read the stored linear predictor as a LOG ODDS regardless of which link put it there, so only a logit was right. It now callsdpar_log_complement(), which is told the link.At the default
link_phi = "logit"nothing moves. Old and new are bit-identical at all 15 linear predictors tested from -700 to 700, maximum absolute difference 0, so no logit result anyone has fitted changes.Away from the default the old density was wrong, per observation, at
eta = 2withtrials = 10,mu = 0.4,y = 0:link_phiphi old error, nats "logit"0.88080 0 "cauchit"0.85242 +0.0303 "probit"0.97725 -0.0965 "cloglog"0.99938 -0.1174 "identity"0.05 (at eta = 0.05)+1.5716 Every one of those constructs today:
bcm_contaminant()passeslink_phistraight tofrmtmb_family(), which does not restrict it per dpar."identity"is the worst case and also the one the framework’s own unit-interval link set allows, where the old code read a probability as a log odds: 1.57 nats per observation atphi = 0.05, falling to 0.19 atphi = 0.5. The file is a shipped example rather than an exported family, so no fitted model in the package’s own tests moves; the Bayesian Cognitive Modeling tier is 363 passing before and after.
The student-t log density loses its digits as nu runs off
RTMB::dt() sends a double straight to stats::dt(), which is accurate, and an AD number to a tape that forms lgamma((nu + 1) / 2) - lgamma(nu / 2) as written. Every fit runs on the tape, so the accurate branch was the one no fit used. The two values agree in every leading digit once nu is large: at nu = 1e10 each is about 2.3e11, where a double is spaced 3e-5 apart, and the difference they must produce is 11.5. Against a 300-bit reference the subtraction is wrong by 1.4e-5 at nu = 1e10 and by 57 at nu = 1e50.
Data with no heavy tails asks exactly this of the density, because a student-t reaches the gaussian only as nu goes to infinity. Past about nu = 1e7 the objective was noise, its gradient changed sign at points the true likelihood is monotone through, and the optimizer stopped at whichever sign change it reached first. That is what the grammar fuzz tier recorded at 0.54.0 as reml-ar1-se-two-optima: the same model fitted to the same data with its rows permuted landed 5.938 apart in parameter space with a logLik gap of 0.004862, 28 times the tolerance derived from the run. It was never two optima.
-
The difference is now formed without the cancellation, by pushing
aup 12 steps with the Gamma recurrence until Binet’s remainder series holds and then taking the Stirling difference in a shape whose large terms are never built. Accurate to 1.3e-14 relative over 369(a, s)points, and 6 ulp end to end over the(nu, z)grid up tonu = 1e50, rising to 2.0e-14 absolute bynu = 1e300, where the head and the normalizing term cancel down to the size of the answer.It is not free, and where it costs is the distributional case. With a scalar
nu, which isnu ~ 1and the common model, the objective measured between 0.75 and 1.33 times its old cost over three independent runs on this machine, with a control tape built from the same density putting the noise floor at 10 to 20 percent. The sign is not settled and the size is small either way. The mechanism is two effects that nearly cancel: the same formula written out by hand is already close to the cost ofRTMB::dt()at a scalardf, and the recurrence then adds a little back.With
nu ~ xit is 2.1 to 3.3 times the old cost, becauseais then a vector of lengthnand the 12 fixed recurrence steps become 12 vectorlog1pnodes where the old code had twolgammanodes.bf(y ~ x, nu ~ z) + student()is a supported model, so this is a real cost, not a hypothetical one. The recurrence length cannot be shortened by readingnu, which is a parameter the optimizer moves after taping. What does shorten it is the Binet series, which depends on no parameter at all: six terms reach 1e-16 atx = 12where four needx = 28, so the shift is 12 and not 25. That halved the vector-path cost, and the sweep behind it found no accuracy ordering between the two.dev/reviews/2026-09-08-remlopt.mdhas the numbers.The permuted and unpermuted fits now agree: the log likelihood is BITWISE identical where it was 0.004862 apart, and the parameter vectors agree to 1.8e-11 where they were 5.938 apart. The residue is entirely in
nu, the parameter that has no maximum; the twothetaagree to 1.2e-16 and 0, andmuto 5.6e-17.frm_allfit()on the same fit went from a logLik spread of 90.1, with one optimizer reporting an impossible -53.05, to a spread of 4.2e-06 across all four. Thereml-ar1-se-two-optimaentry is gone from the fuzz tier’s pending list and the tier is green without it. The multivariate-t carried the same subtraction in two more places, the
gr(dist = "student")covariance blocks and thestudent()fit ofar(cov = TRUE). Both are fixed with the same helper, and thelog(1 + q/nu)in the autocorrelation path is nowlog1p(), which its sibling incovstruct.Ralready argued for. A t block reaching its gaussian limit is how a user checks that it reduces to one, and atnu = 1e16that check was off by 49 log units. It is now exact to the double that carries it.No coefficient moved. Across the reported case the
muestimates agreed to 3.2e-09 before the fix and to 5.6e-17 after: the error was confined tonu, which no user reports, so no scientific answer changes. What changes is that the fit is reproducible.
diagnose() names a distributional parameter with no maximum
A student() fit on data with no heavy tails reports nu in the billions with a standard error in the thousands, and 0.54.0’s diagnose() answered “No convergence problems detected”. The likelihood has no maximum in nu there, so the number reported is where the optimizer stopped.
-
diagnose()gainsunbounded_dpar, which names a distributional parameter whose estimate is far out on its link AND whose standard error is larger than the estimate itself, prints the natural-scale value alongside, and says to refit withgaussian()or to holdnufinite with a prior. The pair is the evidence, not either half: a dpar legitimately far out on its link keeps a small standard error. Measured over 84 student fits (seven error laws,nof 60 and 200, six replicates each),se(log(nu - 1))ran from 4878 to 1.2e4 on the 35 fits whosenuran off and from 0.35 to 22 on the 49 where it did not, with|log(nu - 1)|below 5.1 on every one of those. The two conditions together fired on none of them. Replicated independently on 10 further error laws and 120 fits: 0 of 99 identified fits flagged.READ THE SIGN.
nuruns off BOTH ends oflogm1, and the check catches both. A large POSITIVE estimate means no heavy tails and the fit is the gaussian one. A large NEGATIVE estimate, with a natural-scale value that prints as1, means tails heavier than any identifiednucan hold, and theregaussian()is the worst answer available: on Cauchy errorslog(nu - 1)lands between -20.5 and -17.7 with a standard error of 4003 to 8533, on 8 of 8 fits that reached the boundary. ?frmtmb-familiesgains a “Degrees of freedom that run off” section saying the same thing where a user ofstudent()will find it, andvignette("diagnostics")gains “A distributional parameter with no maximum” beside its “NaN standard errors” section, which is wherediagnose()’s own message already sends people. Both name the two directions and what each one means.
Three defects recorded against the 0.54.0 addition-term work, fixed. Two of them are one guard: the se() scale rule could not tell a second scale from a shape, and a dpar formula walked past it.
A family can now say which dpar a known standard error replaces:
frmtmb_family(se_dpar = "tau")maps outtauexactly as the convention maps outsigma, andse_dpar = NAsays it replaces none, because the known standard error IS the family’s whole scale. That is the case 0.54.0 over-refused. A family with nosigma, another free dpar, and no declaration is still refused, because the core cannot tell a second SCALE, which would be left free and unread, from a genuine SHAPE, which has to stay free. Measured on both: the same data, the samese()column, one family whose extra dpar is a second scale and one whose extra dpar is a skew. The second scale still refuses (before the guard existed it fitted with the scale frozen at its start value and all three standard errorsNaN); the skew, declared withse_dpar = NA, fits at a log-likelihood of -285.459 with every standard error finite, the skew estimated at 2.956 (standard error 0.352) where the data were drawn at 3, and 80.06 log units above the same family with the skew pinned symmetric. The refusal now namesse_dparas a fourth way out.predict(dpar = )reports a declared and mapped-out dpar as the zero it is, which it did forsigmaalone before.sigma()is unchanged: it reports the dpar namedsigma, and a family without one still gets 1.-
A dpar FORMULA no longer walks past that guard.
se()withoutsigma = TRUEsays the residual scale is known, so the dpar it replaces stops being estimated, and only a CONSTANT pins it:sigma ~ 1estimates an intercept the density never reads. The guard now reads every route a formula arrives by,bf(),lf(),nlf()and a family’s owndefault_forms, and refuses the pair by name. This was pre-existing and the BUILT-IN gaussian shared it: measured,bf(y | se(sev) ~ x, sigma ~ 1)fitted at exactly the reference model’s log-likelihood, -62.4977718, with the same two coefficients to seven figures and every standard errorNaN. The fit itself said nothing; the flat direction surfaced only whenvcov()orsummary()was asked for it. The three documented escapes are unchanged and measured again: no formula at all, a constant (sigma = 1), andse(x, sigma = TRUE), which takes asigmaformula as it always did, because the density reads the dpar there.It is a breaking change for a family written against 0.54.0 that declares
se(), has nosigma, and gave its extra dpar a formula: that model fitted before and is refused now. The refusal names the one-argument fix. The discrete censoring convention announces itself ONCE per session. frmtmb reads a censoring bound on a count as INCLUSIVE, so right censoring at
kisP(Y >= k)and every lower edge enters the CDF asF(k - 1); brms emitsP(Y > y)for right and interval censoring of a count. On 200 poisson draws atlambda = 4right censored at 6 the two readings differ by 21.8 to 33.3 log units, median 29.4 over 40 seeds, which is about 0.66 per censored row. (The 0.54.0 entry below gives 20.8 for the same setup; that figure is below all 40 of those draws and should not be quoted again.) The notice fires only on the combination that diverges, a DISCRETE family with RIGHT or INTERVAL censoring, and says the rule, the remedy and where the argument lives. Left censoring, a continuous response and truncation alone are silent, measured. It is a message, sosuppressMessages()silences it, andoptions(frmtmb.notices = FALSE)silences every notice for a session. A call a CENSORING guard refuses, such as one with a half-integer bound, does not spend it; a call refused later, for a reason that has nothing to do with censoring, does. The package’s own suite sets the option off intests/testthat/setup.R, so no test depends on which file censors a count first.The warning from a capped importance correction now says which of two things happened, because they want opposite advice. A correction that is short of rounds shrinks its move by a factor of three to ten each round, and raising
frmtmb_control(importance_rounds =)lands it: that message is unchanged. A STALLED correction takes the same step every round, and its total shift is that step times the round count, so more rounds buy a proportionally larger number rather than a better one. It now says so, and sends the reader toVarCorr(), because a variance component the Laplace fit has already collapsed is what does this. Measured: twelve groups of three Bernoulli rows with no variance component walk at 0.909073 for all five rounds and report a shift of 4.545, and the step is a property of the DRAWS rather than of the data. It changes to 0.317931 atimportance_seed = 7, and at one seed it varies less between two datasets, or between core and afrmtmb.learnfit of an unrelated family with the same group and draw counts, than it does between the rounds of a single run.New
frm_hazard_reads(), a testing aid for frmtmb and its extensions. It reports every place in a package where$is used on one of the containers whose slot names collide under partial matching, so an extension asserts the rule on itself, in its own suite and under its ownR CMD check, from the one container list frmtmb owns. Until now the rule was policed only from frmtmb’s suite, andfrmtmb.couplingpassed its own check and its own tests with 34 such reads.whittle()no longer refuses legitimate tapered periodograms. Its smoothness statistic compares ordinates THREE apart rather than neighbors. A Hann window’s transform is three bins wide, so it correlates neighboring log ordinates (0.31) and leaves ordinates three apart alone (-0.003); at lag one that correlation pulled the statistic from 3.29 to 2.27 and refused about 2 percent of Hann-tapered responses, and at lag three the rate is 0 in 2000 replicates at each of nine cells. Hamming and Blackman windows, which this package does not apply, went from 1.0 and 18.1 percent to 0 the same way. A single Slepian taper, which lag one refused 27 percent of the time at 127 ordinates and 56 percent at 255, is now accepted. The threshold curve is unchanged and the false-alarm rate on the flat null is still 0 in 20000 per cell. Detection of a SEGMENT-averaged periodogram declared raw is unchanged within Monte Carlo error. Detection of leakage costs 1.3 points at exponent 3 and 2.3 at exponent 2.5 with 127 ordinates, and under one point at 255 ordinates and above. What the wider step also costs is an estimate smoothed ACROSS FREQUENCY and then declared raw: a three-bin Daniell smooth of a short record is caught 18 percent of the time at 32 ordinates and 58 percent at 64, where the old step caught nearly all of them, and the two agree again above about 100 ordinates. The refusal message no longer suggests a taper the user may already have applied.
frmtmb 0.54.0
Every link in the registry can now be reached and found: a link on any distributional parameter, a reference page listing all seventeen, and the ordinal families going through the registry rather than a hand-written switch. Making them reachable exposed two silent density bugs. A family written in plain R reaches se() and discrete cens() on its own declaration rather than on its name. A family can declare two addition terms mutually exclusive, which is what an allow-list cannot say. And the frequency domain arrives: frm_periodogram(), whittle(), and a measurement of what an untapered periodogram cannot tell you.
Two gates that made a family written in plain R a second-class citizen of the addition-term grammar. Both were found by the Bayesian Cognitive Modeling port, which had to work around them; the workarounds are gone.
-
se()is no longer gated on the family NAME. A family gets the term by DECLARING that it reads it, withfrmtmb_family(accepts_aterms = c(..., "se"))orrequired_aterms, and reads it asaterms[["se"]].se()is the one core addition term whose entire effect is inside the density, so the declaration is the only thing that can answer whether the term does anything, and a family that does not declare it is still refused, now with a message that says how to opt in. The built-in gaussian and student already declared it and are unchanged.mixture()no longer inherits the declaration from its components: reading the term is half of whatse()means, and the other half maps out a dpar namedsigma, whichsigma1andsigma2are not, somixture(gaussian, gaussian)withse()fitted with both component sigmas at their starting value and every standard errorNaN.The name test was also a silent wrong answer, which the capability test closes. A custom family that merely CALLED itself
"gaussian"and ignored the term was accepted, itssigmawas mapped out, and these()column was discarded: a likelihood with a frozen residual scale and no message anywhere. Measured on main against this branch, same family, same data:ACCEPTED (sigma mapped out: TRUE)becomes a refusal that names the missing declaration. A family that declares
se(), has no dpar namedsigma, and carries another free dpar is refused by name.se()withoutsigma = TRUEreplaces the residual scale, and the core maps out only the dpar the convention names, so the other one would be left free and unread: measured, it sits at its starting value and every standard error isNaN. That is the conditionmixture()is refused for, and openingse()to custom families opened a second route to it. A family whose whole scale IS the known one (no dpar beyond its primaries) is unaffected. Three ways out, all measured: name the scalesigma, pin it in the formula (bf(y | se(s) ~ x, tau = 1)), or writese(x, sigma = TRUE)if it stays estimated alongside the known standard deviation.-
cens()no longer refuses a family for BEING discrete. It still needs a CDF, which is the gate that always made sense. A discrete family that has one is censored under an INCLUSIVE convention: a censoring bound names a value the response can take, so right censoring atkisP(Y >= k), an interval isP(k <= Y <= k2), and every lower edge enters the CDF asF(k - 1). That is the ruletrunc(lb = )has always followed, so one number means one thing on a response however it is bounded.poisson()is censored as a result. Verified against a hand-rolled likelihood for all four codes and composed withtrunc().It DIFFERS from brms for RIGHT and INTERVAL censoring of a count, where brms emits
poisson_lccdf(y | mu), that isP(Y > y). Left censoring agrees exactly, and so does continuous censoring. On 200 poisson draws atlambda = 4right censored at 6 the two readings differ by 20.8 log units.The divergence is deliberate, because brms is internally inconsistent here and frmtmb cannot be both: brms’s own discrete truncation emits an INCLUSIVE lower bound (
lb - 1), so in brmstrunc(lb = 6)meansY >= 6while a right-censored 6 meansY > 6. frmtmb’strunc()reproduces brms bit for bit, so itscens()had to choose between matching brms’s censoring and matching its own truncation. It is also the only reading consistent with the pre-existingsimulate(censored = TRUE), which recordskwhen the latent draw is>= k: 4000 draws from a fit censored at 7 put 0.13250 of the mass at that point, against 0.12190 inclusive and 0.05763 exclusive. The migration vignette carries the argument and the remedy.Two smaller consequences. A one-point interval (
y2 == y) is legal on a discrete response, where it is exactlyP(Y = y); it stays refused on a continuous one. And a non-integer censoring bound on a discrete family is refused by name, because theF(k - 1)shift assumes the unit integer lattice. residuals(type = "osa")is refused on a censored DISCRETE fit, by name. Inclusive bounds put an uncensored row’s support on[lo + 1, hi - 1]rather than the[lo, hi]the one-step window is built on. Every other residual type works, anddharma_residuals()covers the same ground. The link registry became reachable and findable. It has held seventeen links since the last round; almost none of them could be named from the grammar or found in the help.New
frmtmb-linkshelp topic documents every link: what it maps, its inverse, the range it maps onto, which families accept it for the mean, which set each distributional parameter admits, and which links carry a robust field and what that buys. Nothing inman/mentioned probit, softplus, cauchit, squareplus or softit before. Every family constructor’slinkargument,frmtmb_family(links =)and the unknown-link error now point at it, and it is in the pkgdown reference index.Family constructors take a link for EVERY distributional parameter, not only the mean, following brms:
student(link_sigma = "softplus", link_nu = "identity"),zero_inflated_poisson(link_zi = "identity"),skew_normal(link_alpha = "log"). 23 exported constructors gained one argument per non-mean parameter, 34 arguments in all, and the accepted set for every one of the 34 is brms 2.23.0’s set exactly. Each is validated against what that parameter’s support admits, and an out-of-range link is refused with the parameter named. The density is now invariant to which link produced a given parameter value: over all 37 family-by-parameter pairs, holding the natural value fixed and feeding each allowed link’s ownlinkfunof it, 34 agree to a spread of exactly zero and the rest to 1.8e-15.New
frm_family()names a family and its links for the four families ‘stats’ owns, which have no frmtmb constructor to carry the new arguments:frm_family("gaussian", link_sigma = "softplus"),frm_family("poisson", link = "softplus"). It is the analogue of brms’sbrmsfamily(). frmtmb still does not shadowgaussian(),poisson(),binomial()orGamma().cumulative(),sratio()andcratio()go through the link registry instead of a hand-written switch, so they now takecloglog,cauchitandprobit_approxas well aslogitandprobit, andcumulative()also takessoftit. This is brms’s roster exactly.acat()still takeslogitalone, and it is the one place frmtmb takes less than brms. The reason is the density, not the link: probit, cloglog, cauchit and softit all map onto (0, 1) and brms accepts every one of them foracat, but off the logit brms computes a category probability from a SECOND expression, a product of distribution functions times a reversed product of survivals. That expression agrees with acat’s log-linear form when the distribution function is logistic, so it generalizes the same model rather than replacing it, but it has to be written and taped, and substituting a distribution function into the log-linear form does not reach it.acat()’s refusal says all of this, and thefrmtmb-linkstopic records the gap in its table beside the families that do take those links.The ordinal log-space density generalized from the logit to every link that carries an exact log odds. Without it
cloglogandprobit_approxwould return a log density of-Infwith a dead gradient past a linear predictor of 2. Against a 500-bit reference the generalized form holds 1e-13 or better out to a linear predictor of 25. The logit path is arithmetically unchanged.Ordinal thresholds now start on the fitted link’s own scale rather than always on
qlogis.summary()andprint()name the link of every distributional parameter on aLinks:line underFamily:. A coefficient is reported on the link scale and the family name no longer says which scale that is. An ordinal fit reports the distribution function its thresholds are read through, notmu = identity, because the distribution function applies totau - etaand never toeta. A multivariate fit names each response, and the line wraps between entries rather than inside one, which a mixture needs: two components run to 106 characters unwrapped.BEHAVIOR CHANGE,
cumulative(link = "probit"). It now takes the log-space path the logit already took, so a saturated threshold pair is scored exactly instead of through a difference of two rounded distribution functions. Estimates move by the size of that error, which is 1.6e-04 in the log density at a linear predictor of 8 and unbounded beyond it.BEHAVIOR CHANGE,
cratio()’s robust branch read its distribution function attau - etaand relied on1 - F(-x) = F(x). Measured overx = -2, -0.5, 0, 0.5, 2, that identity holds to 1.4e-16 for the logit, 5.6e-17 for the probit, 1.3e-16 for probit_approx and exactly for the cauchit, and FAILS for the cloglog at 2.6e-01: atx = 0it gives 0.36788 against 0.63212. It now reads ateta - tau, which is where the density actually evaluates. No logit result changes. Against a 500-bit reference on a four-category cratio, the naive generalization that keeps reading attau - etahas relative error 1.000 on the cloglog and is exact on every other link; the form that shipped is 1.1e-16 on all of them, at linear predictors where the plain path returns negative infinity.-
Two latent bugs were fixed on the way, and they are the reason this work is more than a grammar addition. Neither could fire before, because the argument that triggers each did not exist.
log_dpar()handed back the linear predictor AS the log of a positive parameter, which is true only on a log link. Withlink_shapenow an argument,negbinomial(link_shape = "softplus")would have read a softplus predictor as a log and returned a wrong density with no warning and no error: ateta_shape = 0.5, soshape = 0.974077, the old code is off by up to 0.339 nats at one point and 0.608 overy = 0:6, and its answer is reproduced exactly by the misread it makes. The shipped version agrees withdnbinom()to 8.9e-16.gate_logs()had the same flaw for a unit-interval parameter, and read a probability as a log odds:zero_inflated_poisson(link_zi = "identity")withzi = 0.3used 0.5744 instead, an error of 0.533 nats. It is now exact to 2.2e-16. What a family declares about the addition terms it reads: the exclusivity the 0.53.0 allow-list could not express, the compatibility cells those declarations already settle, and a structure slot that had no reader. -
frmtmb_family(exclusive_aterms =)declares that a set of addition-term values say the SAME thing to the density, so at most one of them may be supplied. An allow-list cannot express this, because both spellings are legitimately on it:wiener()reads its boundary indicator fromdec()and falls back tovint1, sort | dec(u) + vint(1 - u) ~ 1passed every guard and fitted with a log-likelihood bit-identical to thedec()-only model, the two columns contradicting each other and nothing saying so. Frame assembly now refuses the combination by name, says which spelling the density reads and which to drop.Written in VALUES, as
required_atermsis, so the two compose: an any-of group in one and the same set in the other read together as “exactly one”. It is opt-in rather than implied by an any-of group, becausegddm()genuinely readsdec()andvint1together, as the boundary and the condition index. Declaring a set exclusive whose valuesrequired_atermsdemands TOGETHER is refused at construction, because no model could then satisfy the family. Amixture()keeps a set only if every component declares it, which is the intersection where the allow-list is the union. -
frm_compat()derives its addition-term refusals fromfrmtmb_family(accepts_aterms =)instead of leaving themuntested. A term outside a family’s allow-list is refused BY NAME at frame assembly, sountestedwas never right for that cell: nothing is missing, the guard exists, and the reason is the declaration. Withfrmtmbalone, 104 cells move fromuntestedtorefused, all of themtrials(),vint()andvreal()against families that do not read them; withfrmtmb.eamalso loaded it is 147, the extra 43 beingdec()and the eam families’ own cells. Nothing else moves at either size: audited before the change, no hand-written row disagreed with any declaration.Only the refusal is derived. A family that ACCEPTS a term has said nothing about whether the pair works, so
untestedstays there. And a pair the hand-written rules already refuse keeps its own note, which usually says more than the declaration does;compat_aterm_rules()is exported so a contributing package derives its own rows the same way, deferring to both its rules and the core’s.r("trials()", "kind:family", "untested")is gone, superseded: the four families that taketrials()are named and the other 32 refuse it by declaration, so the kind-level rule won no pair. A structured family’s generic refusal names the family’s own unit: “its likelihood factorizes no finer than a hidden-Markov sequence” rather than “does not factorize over the rows of the data”. That sentence is what
frmtmb_structure(unit =)was documented for and had no reader for; it was also false of a structure that carries nologlik, which is a capability declaration on a family whose likelihood IS rowwise, and those now get a sentence that is true of them.
Four usability defects of hierarchical NONLINEAR models, found while fitting bf(..., nl = TRUE) with random effects on the nonlinear parameters.
A nonlinear parameter named after one of the family’s own distributional parameters is refused BY NAME.
bf(I ~ mu - chi * logw, mu ~ 1 + (1 | id), chi ~ 1 + group, nl = TRUE)used to die insidemodel.frame()with R’s own “object ‘mu’ not found”, which names neither the parameter nor the collision:muis subtracted out of the nonlinear parameters because the family already owns it, so the body’s reference to it was asked ofdataas a column. The refusal says which name, which family, and what the family reserves, and it comes from the parse, sopar_template()refuses it too. brms 2.23.0 on the same formula silently discards the nonlinear body and fits an intercept-only model.A nonlinear body that names its OWN parameter with no column behind it is refused the same way, wherever the name comes from, including
nlf(sigma ~ sigma + 1). A realdatacolumn of that name still wins, so a body reading a column calledmukeeps fitting, and so does a body reading ANOTHER dpar’s per-row value (nlf(sigma ~ ls + th * log(abs(mu)))), which is a deliberate extension and not a collision.startandnewparamsname the collision when a nonlinear parameter is named after a parameter-template component (beta,betad,b,theta,thetaac,thetar,miss). Such a model FITS; only the start list is ambiguous, and it was resolved silently in favor of the component.start = list(b = 1)on a model with a nonlinear parameterbreported “start$b must have length 120”, which describes the random-effect vector, an object the caller never meant. It now says so, and gives the spelling that reaches the parameter (start = list(beta = c("b_(Intercept)" = 1))).NEW
fixef(object, flatten = TRUE): one named vector in thevcov()/confint()/par_template()spelling,dpar_columnwith the location parameter’s own coefficients unprefixed.hypothesis()is a third vocabulary and is NOT included in that claim: it strips parentheses, so it takessigma_(Intercept)only backquoted and spells itsigma_Interceptitself.unlist(fixef(fit))is NOT that vector and never was: it is base R’s composite of a list KEY and a design-column name (mu.x), differing from the model’s own name both in the separator and in namingmuwhere the model does not.?fixefsaid the two agreed. Theunlist()spelling does not move: 19 files in this repository index literaldpar.columnnames out of it, across core and five extensions, and it is base R’s composite rather than a name this package assigns.BEHAVIOR CHANGE, and what changes is the NAMES of
frm_bootstrap()’s default statistic, frommu.xtox: the defaultFUNis nowfixef(f, flatten = TRUE), soapply(bs$t, 2, sd)andsqrt(diag(vcov(fit)))line up by name, which is what that comparison is for.confint(bs)row names change with it. AFUNyou supplied yourself is untouched.Every
ranef()block is addressable. A nonlinear model with(1 | id)on three parameters returns three blocks all namedid, andranef(fit)[["id"]]reached the first one silently.$and[[now also take the BLOCK LABEL,ranef(fit)[["chi: 1 | id"]], which is the"term"attribute each matrix already carried,VarCorr()’s key and thegrpcolumn ofas.data.frame(). A bare grouping-factor name that several blocks share is refused and names the labels instead of answering with the first. What breaks isranef(fit)$gon a factor carrying more than one block, which returned the first block and now errors. Three ordinary spellings do that, so this is not a nonlinear-model corner: an uncorrelated slope(1 + x || g), which desugars to the two blocks1 | gand0 + x | g; the same two written out as(1 | g) + (0 + x | g); and a random effect on more than one distributional or nonlinear parameter, such asbf(y ~ x + (1 | g), sigma ~ (1 | g))or(1 | id)on three nonlinear parameters. A CORRELATED slope(1 + x | g)is one block and still answers to$g. The 0.52.0 grouping-factor keying is unchanged, positional indexing is unchanged, and frmtmb.sample’sranef()on a draws object, which indexes by position, needs no change.diagnose()names FLAT DIRECTIONS: outer parameters with zero gradient and an empty Hessian row. A Gaussian peakexp(lamp) * exp(-0.5 * ((w - pk) / exp(lsig))^2)whose centre starts at 0 whilewruns from 1 to 45 underflows to zero everywhere, so its amplitude, centre and width have no gradient at all; the covariance then fails and EVERY standard error isNaN. The old report listed all nine parameters as having a bad standard error, which names none of them, and the warning asserted overparameterization. The model is not overparameterized. The emptiness of the row is measured rather than assumed, by perturbing the parameter and seeing whether the gradient moves at all (fit$obj$he()is unavailable on a model with random effects), and the check is gated on a covariance that already failed, so a healthy fit pays nothing. Thevcov(),summary()and fit-time warnings name those parameters in place of the overparameterization guess.vignette("frmtmb")gains the starting-value rule for bump-shaped terms andvignette("diagnostics")the two causes ofNaNstandard errors.
The frequency domain. A spectral fit was already possible, because the Whittle likelihood IS an exponential GLM on the periodogram with a log link and exponential() is mean-parameterized, but nothing said so and nothing prepared the data. vignette("spectral") carries the worked AR(1) and power-law fits.
NEW
frm_periodogram()turns one or more series into the data frame a spectral model reads: one row per retained ordinate, with the frequency in Hz. The zero and Nyquist ordinates are DROPPED, so a series of lengthmgivesfloor((m - 1) / 2)rows. Checked againststats::spec.pgram()attaper = 0, detrend = FALSE, fast = FALSE: every ordinate agrees to a ratio of exactly 1 and every frequency to 0, which pins the scaling and the frequency axis together. It offers detrending, Hann and split-cosine tapers, Bartlett segment averaging, and takes a matrix of series or agroupvector of unequal lengths.NEW
whittle(tapers = )names the assumption a spectral fit makes: ordinates independent, exponential about the spectrum. It isexponential(log)at one taper andGamma(log)with the shape FIXED attapersabove it, through a new family slot that reaches the existing constant-dpar mechanism, so an averaged periodogram spends no parameter on a number the data preparation already decided. On an AR(1) of 1024 points atphi = 0.7it returns 0.708617 (se 0.02218) againstarima(method = "ML")’s 0.707178 (se 0.02204).whittle()refuses what the response alone can prove, and states what it cannot see. A response that is not positive is log power or power in dB, which is the common wrong response and is caught exactly. A response too SMOOTH to be raw is caught becausevar(diff(log(I)))is2 * trigamma(k)for ordinates of shapekWHATEVER the spectrum is, and both smoothing and reordering only add to it, so a value far below the model’s is one-sided evidence. It refuses 98 to 100 percent of periodograms that were segment averaged without saying so, and 89 to 100 percent of leakage-dominated ones, against a false-alarm rate measured at 0.000 percent under the flat null over 20000 samples per cell and 0 percent on AR(1), AR(2) and 1/f-with-a-peak spectra. Two limits are documented rather than hidden: four averaged ordinates below about 200 rows are only caught 46 percent of the time, and a near-unit-root AR(1) fires occasionally because its spectrum really is leakage-dominated. A Hann-tapered periodogram is legitimately raw but its neighbours correlate, so it is refused about 1 to 2 percent of the time; the message says so rather than prescribing a taper the user may already have applied.NEW
frm_series_draw()draws a time series from a fitted spectrum.simulate()is refused on a whittle fit instead of returning ordinates that nothing in the object identifies as ordinates.MEASURED, and the practical result of the work: an untapered periodogram cannot estimate a spectral exponent steeper than about 2, at any length. On a cut power law of 300 replicates, true exponents of 2 and 3 both come back between 1.91 and 1.97, because the leakage floor of an untapered transform falls like
f^-2and anything steeper sits under it. A LONGER recording is monotonically worse: the bias at exponent 3 goes from -1.033 at 256 points to -1.094 at 1024. A Hann taper removes it (0.087 and 0.029). At an exponent of 1, flatter than the floor, the raw estimate is unbiased, which is why this is a taper argument and not a taper default. Whittle’s own small-sample bias inphiis separately smaller in magnitude than exact Gaussian ML’s in every cell measured at 64 and 128 points, and comparable above.
frmtmb 0.53.0
set_prior()’s own class names mean what they mean in brms; the structured-family protocol factorizes a whole-response log-likelihood per row and per group, so the importance correction, loo() and deviance residuals reach hmm() and the learning families; every brms link name is in the registry; car(type = "esicar") prediction errors are exact; a Bayesian Cognitive Modeling tier fits fifty-two of the book’s models against their Stan programs; and frmtmb.learn, the sixth extension, brings reinforcement-learning families.
frmtmb_register_compat(expects = )accepts a name another package already supplies. It used to refuse one, so frmtmb.sample, which expectshmm, failed to load in every session that had loaded frmtmb.latent first. An expectation the session has already met is met; only a name the same call supplies infeatures =is refused, since that declaration contradicts itself.BUG FIX.
mixture()started every component at a quantile of the raw response. For a bounded mean that is a count outside the logit’s range, so every component fell back to the link origin, where they are the same distribution, andmixture(beta_binomial, beta_binomial)on the malingering data of Lee and Wagenmakers sat at two identical components (logLik -64.26) with no warning where a separated start reaches -58.71. A component whose mean link is on (0, 1) now starts from the quantile ofy / trials, clamped to (0.02, 0.98) the way the component families clamp their own start, because eight of those twenty-two respondents score 45 of 45 and put the two-thirds quantile at exactly 1. Found by the Bayesian Cognitive Modeling port.residuals(type = "deviance")on a structured family that declaresloglik_rowbut nofitted_meannow says which half is missing (the sign of each row’s departure from its conditional mean) instead of claiming the family has no unit deviance and listing thirty built-in families that do.car(type = "esicar")standard errors no longer carrycon_sd.expand_b()centers the field each connected component contributes, so the linear predictor seesP bwithPthe centering projection, but the delta method paired the design columns withbthroughdc/db = I. Everypredict(se.fit = TRUE)standard error and everyranef(condVar = TRUE)conditional SD therefore carried the inert component means, exactlycon_sd^2in the variance: measured 1.348e-5 relative in a standard error at the 1e-3 default, 1.347e-3 atcon_sd = 0.01, and 7.5 to 12.7 percent atcon_sd = 0.1, growing a hundredfold per decade. The Jacobian is now the projection itself, so an esicar block’s contribution isZ P V P' Z'andPannihilates that coordinate whatever its scale. On a 4 by 4 lattice the standard errors drop by 1.348e-5 relative and the variance by exactlycon_sd^2; a singleton component’s conditional SD was exactlycon_sdand is now exactly zero, which is the uncertainty of a field value that is exactly zero.logLik(), the estimates,ranef()’s values andVarCorr()are bit-identical, and so is every reported number forescar,icarandbym2. This closes the residual the 0.52.0 entry below documented and theesicarreview recorded as R2. Prediction standard errors for a reduced-rank (rr) block are unaffected: they already went through this Jacobian.conditional_effects()on a fit reads its display quantity from a newce_pred_dpar()rather than deriving it inline. Same answer, same models: the expression moved, andmean_displayis nowis.null(pred_dpar), which it always was. The point of moving it is that a sampling extension drawing the same curves per posterior draw has to make the same choice, and could not:mean_is_mu()is a structural test of a family’smean_fnand stays private.The conditional-effects seam of
?"frmtmb-sampling-api"gains eight exports, so that a sampling extension can return the frame this package returns instead of assembling a second one:ce_frame()(the column set and its order),ce_display_kind()(which of the three displays a call asks for, and the refusalscategorical =owes),ce_group_vars()(the grouping variables are_formula = NULLcall blanks), andce_new_level_spec(),ce_boot_grids()andce_draw_new_levels()(the placeholder level and the draw that makes an unobserved group carry its own effects),ce_pred_dpar()(what gets predicted, which is not always the dpar the display is labeled with) andce_dots()(theallow_new_levelsspellings aconditional_effects()call accepts). All eight were already what the fit method used;frmtmb.samplewas reimplementing five of them, four badly. See that package’s NEWS for what the exports bought.frm_lp_basis(newdata = )now makes the same statement about aps()knot span thatpredict(newdata = )makes. The two doors evaluate the same closures at the same points, and only one of them said that the basis stops being a partition of unity past its frozen outer knots; a curve drawn through the basis seam was silent about a cliff the fit-end report exists to warn about. The warning fires ONCE perps()term per call, whatever the grid width or the coefficient count, and its text is byte-identical topredict()’s on the same fit and grid. In sample (newdata = NULL),fitted()andsimulate()stay quiet, as before.FIX: the span check could not run on a taped body at all. Its guard was
is.numeric(), which is TRUE for an RTMB advector, so aps()term whose argument names a nonlinear parameter would have raised “Comparison is generally unsafe for AD types” as soon as any taped caller armed it. The guard isinherits(x, "advector")now, and the basis seam additionally evaluates the body once OFF the tape, so that case is checked on numbers rather than skipped.The
ps()span warning carries the condition classfrmtmb_ps_span_warning, so a consumer can catch, muffle or escalate this one warning without matching on its text.frmtmb.splineuses it to surface the warning once under its own function names and to refuse a feature search whose bracket leaves the span.-
BEHAVIOR CHANGE.
set_prior()’s own class names now mean what the same names mean in brms. A distributional parameter’s own name is a class here as it is there:set_prior("student_t(3, 0, 2.5)", class = "sigma")is a density on sigma ITSELF, on its natural scale, with the inverse link’s log-Jacobian applied. The two brms spellings are mutually exclusive by model shape, measured offbrms::make_stancode(), and frmtmb now refuses each on the other’s model, by name, saying which one applies:the model the spelling it takes bf(y ~ x)class = "sigma", a density on sigmabf(y ~ x, sigma ~ 1)orsigma ~ xclass = "Intercept", dpar = "sigma", a density on the log-scale interceptWhat changes in your numbers. Before this release,
set_prior(..., class = "Intercept", dpar = "sigma")was accepted on a model with nosigmaformula and meant a density on LOG sigma. On a gaussiany ~ xwith 200 rows whose maximum-likelihood sigma is 2.0103,normal(0, 0.5)gave 1.9965 that way; the same script now stops, andclass = "sigma"gives 1.9429, which is whatbrms::prior(normal(0, 0.5), class = "sigma")gave all along. To reproduce the OLD placement, writesigma ~ 1inbf()(that model has a log-scale intercept, and the same spelling addresses it), or reach the raw parameter withprior = list(betad = ). Aclass = "b"prior on a dpar’s slopes is unaffected: it was link-scale before and is link-scale in brms. get_prior()lists a distributional parameter under whichever of the two spellings its model offers, which is the row brms lists: class"sigma"where sigma has no predictor,class = "Intercept"withdpar = "sigma"where it has one. A parameter with NEITHER spelling, because its class is one frmtmb refuses by name, is left out rather than advertised: a mixture proportion with no predictor of its own was briefly listed as class"theta1", whichset_prior()refuses.set_prior()refusesclass = "b", dpar = <name>where that parameter’s predictor is an intercept only. There are no slopes for the row to address there, so it used to be accepted and applied nothing at all, silently; brms refuses the same row.set_prior()refuses acoefor agroupon a distributional parameter’s own class. The class names one parameter, so neither argument had anything to narrow to, and both used to be accepted, printed, and then dropped while the prior applied anyway.print()on a prior list, and the slot names inside prior error messages, spell a density on a distributional parameter with that parameter’s own class:student_t(3, 0, 2.5) class=sigma scale=naturalwhere a translated row used to printclass=Intercept dpar=sigma scale=natural. What is printed is whatset_prior()accepts on that model, which is the promise the printer makes.frm_sample()’s default-prior announcement uses the same spelling.The brms translation is now a pass-through rather than a second set of rules: a translated row keeps its class and
set_prior()decides the placement, sofrm(prior = brms::prior(...))andfrm(prior = set_prior(...))are one code path and cannot drift. The priors tier fits the same model both ways and asserts the objectives are identical to 1e-12, class by class.set_prior()refuses acoefon class"sd"or"cor", which it used to accept, print, and then apply to the whole block. The translated route already refused it; both do now.A prior class frmtmb keeps somewhere else (
sds,sdgp,lscale,simo,sdcar,car, brms’s mixturetheta1) is refused by name fromset_prior()too, with the same sentence the translated route gives, rather than being read as a distributional parameter that the model does not have.No
natural =orscale =argument was added: after this change the placement follows from the class, which is brms’s own rule, so there is nothing left for such an argument to override.New
vignette("bayesian-cognitive-modeling"): a reference list of the models in Lee and Wagenmakers, Bayesian Cognitive Modeling, each with its frmtmb spelling and how it was checked against the Stan port of the same model.tests/testthat/helper-stan.Ris the harness the checks run through: give it a Stan program, its data and a map from an frmtmb estimate to its parameters, and it asserts thatlog_probequals frmtmb’s log density there plus a stated constant. The families the book needs and the core does not ship are ininst/bcm/, alongsideinst/rl/, andinst/COPYRIGHTScarries the BSD-3 notice of the Stan programs the tier adapts.
Three seams in the structured-family protocol: a family can say how finely its likelihood factorizes, a family can say which addition terms it takes, and the compatibility vocabulary is built once instead of once per registration.
frmtmb_structure()gainsloglik_rowandloglik_group.loglikreturns one number for the whole response, which is what the objective needs and less than every other consumer needs, so a structured family was refusedfrm(importance = ),loo(),waic()and deviance residuals whether or not its likelihood factorized. The two slots carry the pieces a family HAS, at the finest granularity it has them: one value per row, or one per level of the block’s new reservedgroupentry.unitis unchanged and still declares what may honestly be left OUT, which is a different question: the workedrw_delta()family factorizes per trial and leaves out a whole subject.frm(importance = )corrects a structured family that declares either slot, instead of refusing every one of them. The refusal that remains names the missing declaration. The family’s own units and the model’s grouping levels must be the same partition of the rows and are checked: summing a per-subject likelihood into a per-item proposal is silently meaningless, so it is refused by name. A family’s handling of the stacked design the correction evaluates is verified against the plain objective, per group, by the check that was already there.residuals(type = "deviance")works for a structured family that declaresloglik_rowand attaches its saturated log-density to what that slot returns. Without the saturated half the refusal names it, rather than assuming a value that is right for a Bernoulli trial and wrong for a Poisson count.frmtmb_family()gainsaccepts_aterms, the allow-list that complementsrequired_aterms. An addition term a family never reads used to be parsed, stored on the fit and ignored in silence:lba(3)fittedrt | dec(two) + vint(choice) ~ 1with fixed effects bit-identical to the model withoutdec(). Frame assembly now refuses such a term by name and lists the ones the family takes. The built-in families declare theirs.NULL, the default, accepts every registered term, so a custom family written before this keeps its behavior, and the check runs after every guard that can say something more specific, so no existing message changes.The
frm_compat()feature vocabulary is cached and the cache is extended by each registration rather than rebuilt. An extension registering four addition terms from its.onLoad()paid four 16 ms builds and now pays one.
frmtmb 0.52.0
A brms prior means what it means in brms; conditional_effects() plots the expected response and conditions on a new level; car(type = “esicar”) is the exactly constrained model; the spline seams a curve needs, and ps(), a penalized curve a nonlinear body can consume; and the drift-diffusion extension becomes frmtmb.eam with two more accumulator families.
New
vignette("reinforcement-learning"): a worked example that writes a Rescorla-Wagner delta-learning family for a two-armed bandit withfrmtmb_family()and the structured protocol’s whole-responseloglikslot, then fits it hierarchically through the ordinary grammar. The family is not exported. It lives ininst/rl/rw-delta.R, which the vignette reads withknitr::read_chunk()andtests/testthat/test-rl-example.Rfits, so the page and the test cannot drift apart. The example checks itself three ways: against an independent scalar reference for the recursion, against a Stan program’slog_probat frmtmb’s own estimates (gated onFRMTMB_BRMS_FIT_TESTS), and by parameter recovery over 100 simulated datasets. It also measures what the Laplace approximation costs at 20 trials per subject, and names the one protocol seam it wanted and did not find:loglikreturns one scalar, sofrm(importance =)and a pointwise log-likelihood are out of reach even for a sequential family whose likelihood does factorize. Seedev/rl-findings.md.The drift-diffusion extension is renamed frmtmb.eam (evidence accumulation models); see its NEWS.
-
The AD-safe link registry completes the brms 2.23.0 roster. New
probit,probit_approx,cauchit,softit,softplus,squareplus,sqrt,log1pand1/mu^2links, each reproducing brms’s own definition, sobernoulli(link = "probit")is now the everyday spelling of a probit regression rather than a custom link list passed throughget_link().1/mu^2is the inverse Gaussian canonical link and is unrelated topower12, the tweedie power link. Every link name brms accepts is now accepted here.Six of them carry a robust field. Four of the six move a saturation boundary, and each boundary is measured rather than assumed.
probit’s log-odds is a difference of twopnormlogs, which holds to|eta| = 38where the plain round trip gave out at6.4;probit_approx’s is the cubic itself and never saturates, where the plain path gave out at5.9;squareplus’s log mean isasinh(eta / 2), exact where the sum(eta + sqrt(eta^2 + 4)) / 2has cancelled to zero;softit’s log-odds is the softplus’s log.softplusandsqrtcarry alog_etafor a different reason: it selects the family’s robust log-density, and without itnegbinomial()formsmu + mu^2 / shape, whose excess overmuis lost to rounding at a mean near zero.cauchithas no robust field and needs none: its tails are polynomial, so the plain round trip is still accurate ateta = 3e9.stats::binomial(),stats::poisson()andstats::Gamma()validate their link string throughstats::make.link()beforefrm()sees it, so the four namesstatsdoes not know (probit_approx,softit,softplus,squareplus) have to come in through one of frmtmb’s own family constructors. Seevignette("brms-migration")anddev/links-findings.md. -
BUG FIX.
conditional_effects()returned an INVERTED band on every decreasing link.inverseand1/mu^2send the lower linear predictor to the upper response value, and one of the three places the band’s endpoints are transformed did not order them afterwards, soGamma(link = "inverse")came back withlower__aboveupper__on every row. The endpoints are now ordered by the link’s direction rather than sorted, which also keeps a missing bound on the side it belongs to.A bound the link’s domain does not reach is now
NAinstead ofNaN, with one warning naming the link and the count. Two cases:1/mu^2is1 / sqrt(eta)and has no value once the band reacheseta = 0, so such a band has no upper bound at all; andinversewith the band straddling zero has a pole between two finite ends, where neither end bounds the response. The second was previously reported as an ordinary interval.This surfaced through the new
1/mu^2link, which makesfrm(family = stats::inverse.gaussian())fit for the first time, but the inversion was not new: it reproduces onGamma(link = "inverse")with the link registry untouched.
The ten defects the brms post-fit method tier recorded in dev/brms-methods-tests.md, repaired. Nine of them are in conditional_effects(), which now returns brms’s frame, brms’s grid and brms’s numbers wherever the two packages mean the same thing.
- BEHAVIOR CHANGE.
conditional_effects()draws the EXPECTED RESPONSE on every family. Its defaultmethod = "epred"took the point estimate as the inverse link of themupredictor, which is the expected response only when the family’s mean is that. On a zero-inflated fit it plottedexp(eta)where the mean is(1 - zi) * exp(eta): 15% high at the first grid point and 268% high at the last, with no warning and a smooth curve of the right shape. On a hurdle fit the error changed sign along the curve, 14% below the mean at one end and 237% above at the other.fitted(),predict(type = "response")andconditional_effects(method = "predict")were right all along, so the package disagreed with itself on the plotting path alone. - Which fits move, in full. The gate is
mean_is_mu(), so the default curve changes for fourteen families:asym_laplace,beta_binomial,binomial,cox,hurdle_gamma,hurdle_lognormal,hurdle_poisson,lognormal,shifted_lognormal,zero_inflated_asym_laplace,zero_inflated_beta,zero_inflated_binomial,zero_inflated_negbinomialandzero_inflated_poisson, plus any response carryingtrunc()and anymixture(). Every other family is unchanged, bit for bit, which was checked family by family. -
lognormalwas a defect of its own and nobody had reported it: itsmulink is the identity, so the old path plotted the LOG-SCALE LOCATION as if it were the response. On a fit whose smallest observation is 0.077 the drawn curve ran from -1.489 to 1.454, negative for a strictly positive response, where the meanexp(mu + sigma^2/2)runs 0.286 to 5.423.shifted_lognormal,asym_laplaceandcoxmove for the same reason. - The band on that curve is the delta method over EVERY distributional parameter’s coefficients jointly, so the cross-parameter covariances are in it rather than dropped. It is symmetric on the
mulink’s scale when the mean lives there, on the log scale when the mean is positive, and on the response scale otherwise, which reduces exactly to the old link-scale band wherever the mean is the inverse link ofmu. On a zero-inflated fit its width is within 13% of a 200-refit bootstrap band’s. - BEHAVIOR CHANGE.
conditional_effects()returns brms’s columns: the varied predictor, then every other model variable at the value it is held at, thencond__(always, not only underconditions =),effect1__and, for a two-variable effect,effect2__. brms’s ownplot()facets oncond__, so ported faceting code has something to facet on, and the held values are no longer invisible. - BEHAVIOR CHANGE. A two-variable effect holds its moderator at the EXACT
mean +/- sd, not atsignif(mean +/- sd, 3). Display rounding moved to theeffect2__label, where brms puts it. The curve was the model evaluated at a covariate value nobody chose, and how wrong it was depended on the coefficient rather than on anything visible in the plot. - BEHAVIOR CHANGE. The two-variable grid varies the FIRST effect slowest, as brms’s does, so the two frames now agree elementwise and a script that indexes rows positionally ports.
-
int_conditions =is implemented, with brms’s meaning: a named list of the values (or a function of the observed column) that one or both variables of an effect are evaluated at. It was accepted, ignored, and reported as an unknown argument topredict(), a function the user had not called. - BEHAVIOR CHANGE.
re_formula = NULLconditions on a NEW group, as brms does, and the grouping column of the returned frame saysNA. It used to take the first observed level silently: onsleepstudythat was Subject 308, a curve 86.5 away from the population one at its furthest point, with nothing in the frame to say whose it was and the choice depending on factor level order. A new group’s conditional modes are zero, so the curve is the population curve and the random-effect variance goes into the band instead; brms draws that group’s effects afresh per posterior draw, which is the paradigm difference. To condition on an observed group, name it:conditions = list(g = "3"). - BEHAVIOR CHANGE.
categorical =is honored on an ordinal or categorical fit, and the per-category layout is keyed"x:cats__"as brms keys it. The argument was accepted and did nothing, so the other layout could not be asked for at all, andce[["x:cats__"]]wasNULLon a frmtmb result. The default is unchanged (one curve per category, which is what brms’s own message asks the user to switch to);categorical = FALSEgives brms’s default summary, the expected category numbersum(k * p_k), with a delta-method band of its own. - BEHAVIOR CHANGE. A
mo()predictor gets one grid point per LEVEL, as brms does, instead of the 100-point continuous grid it shared with every other numeric predictor. A monotonic effect is defined at the ordered levels of its variable and nowhere between them, so 96 of those 100 points were the model evaluated where it has no meaning. - BEHAVIOR CHANGE. A
trials()variable is held at 1 on the effect grid unlessconditions =pins it, which is brms’s rule, and the call says so once. A grid row is one artificial observation, so the mean number of trials was not a whole number and the expected count over it was not a quantity anyone asked for.method = "predict"used to refuse the same model outright. -
conditional_effects()finds a covariate that sits on a distributional parameter rather than onmu. Onbf(y ~ 1, theta1 ~ x) + mixture(...)it refused with “No plottable predictors found for dpar ‘mu1’”, naming the one linear predictor it had looked at rather than the model the user fitted. It falls back to every parameter of the response only when the selected one has nothing to plot, so a model whosedpardoes have terms enumerates that parameter’s terms exactly as before. - An unknown argument to
conditional_effects()is reported againstconditional_effects(), once per call. It used to be forwarded topredict()and reported there, and only from the branches that forward: on an ordinal or categorical fit an unknown argument was discarded in complete silence. -
conditional_effects(band = "boot", re_formula = NULL)draws the new group’s effects once per bootstrap replicate, so the interval carries the group variance the way the wald band does. It did not: every replicate predicted the new level’s zero modes, so the band was the POPULATION band under another name: on sleepstudy 1.00x the population band and 0.24x the wald band for the same stated quantity, and on an ordinal fit bit-identical to the population frame in estimate AND width, while theNAgrouping column claimed otherwise. It matters most there: the ordinal per-category delta method is refused withre_formulaand the message sends the user toband = "boot", so that was the only band available and it was the broken one. The bootstrap band is now 3.6 to 4.2 times the population band on sleepstudy and lands within 10% of the wald band at 100 refits. Blocks whose levels ARE the structure (gr_cov,gr_prec,car,spde) get zeroed rather than drawn, which is the same assumption the wald band makes for them. -
effect2__keeps one level per distinct moderator value. It took its levels fromround(v, 2), so two values that rounded together became ONE level.plot()groups on that column, so it drew two curves as one series, and the names on anint_conditionsentry were dropped with them. The rounding is the label only; labels widen past two decimals just far enough to stay distinct. -
conditional_effects()accepts brms’smethod =spellings ("posterior_epred","posterior_predict") as aliases;"posterior_linpred"is refused by name, pointing atdpar =. - The expected-response display refuses an addition term whose value on a grid row would be a reference value rather than a real one (
trunc(lb = v)with a variable bound), with the messagemethod = "predict"has always used. It used to average the bound silently. - BEHAVIOR CHANGE. A mixture’s mixing weights have a response scale that is the softmax over the component predictors, so
predict(type = "response", dpar = "theta1")returns brms’sposterior_epred(dpar = "theta1")exactly. It returned the linear predictor: not a probability, and on the tier’s own fit outside[0, 1]on 1.25% of the rows, reaching 1.112. The likelihood is untouched, the softmax having always been applied inside the density, andpredict(type = "link", dpar = )still returns the predictor. Its standard error underse.fit = TRUEis the delta method through that weight’s OWN predictor,p (1 - p), which is exact for two components and conservative for three or more, where the softmax also moves with the other components’ predictors: measured 5.5% to 26.1% wider than the joint delta method on a three-component fit, never narrower.?mixturesays so and a test pins it. - BEHAVIOR CHANGE, and what breaks is
ranef(fit)[["Days | Subject"]], which now returnsNULL: aNULLflows on into arithmetic asnumeric(0)rather than erroring. Code that indexed a random-effect list by the BLOCK label needs the grouping factor instead (or[[1]], or the"term"attribute).ranef()keys its list by the GROUPING FACTOR now, as brms and lme4 do and as this package’s owncoef()already did:ranef(fit)$Subjectused to beNULLin a model wherecoef(fit)$Subjectwas a data frame. The block label rides along in each matrix’s"term"attribute, is whatas.data.frame()puts in itsgrpcolumn, and is stillVarCorr()’s key, so two terms on one factor stay distinguishable.frmtmb.sample’sranef()on a draws object follows core and is re-keyed with it; that package’s own NEWS says so. - BEHAVIOR CHANGE.
predict(type = "response", dpar = "sigma")on ay | se(s)model with nosigma = TRUEreports 0, as brms does and as this package’s ownsigma()already did. It reported the log link’s inverse of the mapped-out coefficient, 1, which reads as an estimate of a parameter the density does not have.
Behavior change
-
car(type = "esicar")is the exactly constrained intrinsic CAR brms fits, and no longer an alias forcar(type = "icar"). brms declaresNloc - 1free values and sets the last to minus their sum; frmtmb keepsNloccoefficients and removes each connected component’s mean on the way to the linear predictor, which is the same model and keeps one coefficient per location forranef(),predict()andsimulate(). The normalizer is the pseudo-determinant,(Nloc - c) log tau + log|K|. A fit spelledesicarchanges: refit it, or spell the termicarto keep the softly constrained model it used to select. The new fit sits ABOVE the old one, and not by much: on a 4 by 4 lattice the log-likelihood rises by 4.0e-4 andsd(car)by 3.1e-5 relative, the bias the soft constraint was always documented to carry. The constraint is what changes exactly, not the numbers: the field now sums to zero to machine precision instead of to about 1e-9. -
con_sddoes not change what anesicarterm FITS. The constraint is exact, sologLik(), the estimates andVarCorr()are invariant to it. It is not inert everywhere, though: the coordinate it scales reaches every delta-method standard error (predict(se.fit = TRUE)) and everyranef(condVar = TRUE)conditional standard deviation, ascon_sd^2in the variance. At the 1e-3 default that is 1.3e-5 relative in a standard error, but it grows a hundredfold per decade, socon_sd = 0.1inflates them by 7 to 13 percent. Leavecon_sdalone on anesicarterm. Onicarandbym2it means what it always did. (The standard-error half of this is no longer true; see the development version’s entry.)
Improvements
- The brms log-density tier gains row 19c-esicar, an identity with brms’s own
sparse_icar_lpdfprogram at the same parameter vector, with the CAR normalizer admitted as a closed form in the data (0.5 Nloc log(2 pi) - 0.5 log det K, residual 3.0e-13). The tier’s exemption list drops from four divergences over three rows to three, one per row;bym2is what is left of row 19c. -
esicaron a disconnected adjacency matrix constrains each connected component rather than the global sum alone, so the field stays proper where brms’s own normalizer does not apply. The two agree whenever the graph is connected.
A brms prior specification now means in frmtmb what it means in brms. Four behavior changes, each measured against the Stan program brms generates: on every shape whose optimum is a real mode, frmtmb’s penalized objective is now brms’s posterior density up to one log(2) per lower-bounded parameter, which is a constant and moves no mode.
- BEHAVIOR CHANGE:
frm(prior = )applies what a brms prior row SAYS, keyed on thepriorstring rather than on the table’ssourcecolumn.frm(prior = brms::get_prior(...))used to apply NOTHING and say so in a message; it now fits a penalized model. The sharpest form of the old behavior was that a row the USER had edited into aget_prior()table in place, the ordinarygp$prior[i] <- "normal(0, 20)"workflow, was dropped too, because brms does not updatesourceafter such an edit, and the message said brms had filled the row in. What to change: a script that passed aget_prior()table for its slot listing and expected an unpenalized fit must now blank thepriorcolumn, or not pass the table. A table containing a class frmtmb keeps elsewhere (sds,sdgp,lscale,simo, brms’s mixturetheta) now stops the call with a named refusal where the row used to be dropped in silence; blank that row, or write the frmtmb class the message names. - BEHAVIOR CHANGE: a brms prior on a distributional parameter’s own class (
sigma,shape,phi,nu,kappa,sigma1, …) is translated instead of refused, and it lands on that parameter ITSELF through the dpar’s inverse link with that map’s log-Jacobian, which is where brms puts it. On a nonlinear fit this reproduces brms’s mode to nine figures where the row previously stopped the call. A bound on such a row travels with it, so brms’slb = 0on a log-linked dispersion becomes no constraint rather than a floor of 1. The brms classesar,ma,cosy,cortimeandrescorare carried over too;set_prior()already spelled them the same way. What to change: nothing in a frmtmb script.set_prior("...", class = "Intercept", dpar = "sigma")still means a density on LOG sigma and is deliberately left alone, so the two spellings now mean different things: writebrms::prior(..., class = "sigma")for brms’s meaning.?set_priorrecords the divergence and its measured size. - BEHAVIOR CHANGE: a
class = "Intercept"prior is evaluated at the intercept at the MEAN of the predictors, which is the intercept brms constrains, rather than at the intercept at zero. OnReaction ~ Days, wheremean(Days)is 4.5, brms’s own default moved the slope by 0.068 standard errors under the old placement and moves it by 0.00003 under this one. Every sub-formula with an intercept is centered separately, as in brms; nonlinear sub-formulas, a smooth’s unpenalized columns andmo()terms are not centered on either side. What to change: an existingclass = "Intercept"prior on a model with uncentered predictors now constrains a different quantity and its fit moves. To keep the old meaning, name the intercept as a coefficient instead:class = "b", coef = "Intercept". - BEHAVIOR CHANGE: an ordinal family’s
class = "Intercept"prior addresses its THRESHOLDS, as the same row does in brms, instead of failing with “Prior target not found”. The density sits on the threshold vector at the predictor means, with the log-Jacobian of the map from frmtmb’s internaltau_raw, and it reproduces brms’s density exactly.get_prior()lists the slot. What to change: a script relying on that error will now fit;prior = list(tau_raw = )still reaches the same parameters on the internal scale. -
frm_simulate(prior = )follows the same placements. A draw from a distributional parameter’s own class is a draw of that parameter, and a draw fromclass = "Intercept"is a draw of the intercept at the predictor means; theparsattribute reports the value that was WRITTEN, so it still round-trips throughnewparams =, whoseInterceptis the intercept at zero. On a model with uncentered predictors that column now holds a different number. - BEHAVIOR CHANGE: a translated
class = "sd"orclass = "cor"row carrying acoefis refused rather than applied to the whole block. brms narrows such a row to one coefficient and writesexponential_lpdf(sd_1[2] | 1), keeping its default on the rest; frmtmb resolves those classes per BLOCK and never readcoef, so every one ofcoef = "Intercept",coef = "x"and nocoefgave a bit-identical objective. Applying a narrowed row is a WIDER prior than the one asked for. What to change: drop thecoefto prior the whole block (groupstill selects which one), or address the one parameter withclass = "theta".coefon class"b", andgroupon"sd"/"cor", are unaffected: those do narrow, and are now pinned. -
logistic(mu, s),gamma(shape, rate),inv_gamma(shape, scale)andbeta(shape1, shape2)parse, withprior_logistic(),prior_gamma(),prior_inv_gamma()andprior_beta()exported beside the other prior constructors. Those are brms’s defaults on a mixture proportion, onshape/phi/nu/kappa, and on a zero-inflationzior hurdlehu. Adding them takes every negbinomial, zero-inflated and hurdle model out of the set of tables that stop: of 20 shapes a brms user would actually write, 6 stopped and 3 do now, all three for a CLASS frmtmb keeps elsewhere (s(),gp(),mo()) rather than for want of a density. - Every refused row of a brms table is named in one message instead of only the first. A
y ~ gp(x)table carries both anlscalerow and ansdgprow and used to cost two edit rounds; it costs one now. - The refusal for brms’s mixture
thetano longer namesclass = "Intercept", dpar = "theta2", a spelling that then failed with “Prior target not found”.sds,sdgp,lscale,sdcarandcarare refused by name as well, each saying which frmtmb class reaches the same parameters, instead of being offered the generic dpar advice. brms’s shrinkage priorshorseshoe(),R2D2()andlasso()now reach that same “unsupported distribution” message;R2D2()used to fall to a generic parse failure because its name is upper-case and its argument list empty. -
tests/testthat/test-brms-priors.Rgrew from 61 assertions to 83 and every divergence it used to pin is now an identity.
Two exported seams an extension needed and could not get, a log survivor function so that right censoring is exact rather than floored, a penalized spline a nonlinear body consumes by VALUE, and two defects found on the way.
- NEW
frm_joint_cov()returns the joint covariance of everything the fit estimates:beta,betad,thetaand the random-effect coefficientsbtogether.vcov(full = TRUE)cannot return it, and is documented not to: its row names are exactlyconfint()’s, sobis absent under both of its branches. A penalized smooth’s wiggly part is a random-effect block even when the smooth is a population term, so a covariance that stops at the fixed effects covers none of it. The result is memoized on the fit and carries alabelselement naming every row. It is also the only route to the covariance of an AUTOSCALED fit: a freshRTMB::sdreport(getJointPrecision = TRUE)goes round the reparameterization and returns a covariance built on the unscaled Hessian. - NEW
frm_lp_basis()returns the design of a linear predictor over the coefficient vector:eta,A(d eta / d coef),coef_pos,V,coef_namesandextra_var.predict(se.fit = TRUE)builds all of that internally and keeps only the diagonal ofA V A'; every delta-method quantity over a fitted curve - a contrast between two grids, a simultaneous band, a derivative, the time of a peak - needs the whole thing.predict(se.fit = TRUE)is now written as a two-line consumer of it, which is the test that the shape is right. -
frm_lp_basis()serves the two cases nothing else could. For a NONLINEAR bodyAis a Jacobian, taped from the body against the coefficients it reaches through;predict(se.fit = TRUE)stays refused there and this is the route. For anrr()block atre.form = NULLthe loadings live intheta, so a design over(beta, b)alone is incomplete;Acarries the loading columns andcoef_posnames theirthetarows. An exactgp()’s kriging variance and a new grouping level’s marginal variance come back inextra_var, separately, because neither is coefficient uncertainty. - NEW
frmtmb_family(lccdf = ), an optional LOG SURVIVOR function. Core formed a right-censored row’s contribution aslog(1 - F), and a double cannot represent the complement of a probability that has rounded to one, so past that point the contribution was not merely inaccurate: it was CONSTANT, with a gradient of exactly zero. An optimizer prices such a row the same however far it moves. A family that declareslccdfis scored from it instead. Measured on a standard normal tail,log(1 - pnorm(z))is-Inffrom z = 8.3 and already 0.068 wrong at z = 8;pnorm(z, lower.tail = FALSE, log.p = TRUE)is exact to z = 500, value and derivative alike. -
gaussian(),lognormal(),exponential(),weibull()andcox()declarelccdf.poisson()does not, becausecens()is refused for discrete families;inverse.gaussian()does not, because RTMBdist’s upper tail is computed on the probability scale and reaches-Infat the samelog S = -34thatlog(1 - F)does.lccdffixes RIGHT censoring and leaves left censoring, interval censoring and truncation on the old footing, so a LEFT-TRUNCATED survival model meets the same problem from the other side. - A family that supplies only
lccdfand nolcdfnow accepts right censoring, and refuses left censoring, interval censoring andtrunc()by name. - NEW
ps(expr, k =, degree =, pad =, center =), a penalized coefficient block whose VALUE a nonlinear body consumes, admitted only insidebf(..., nl = TRUE)ornlf(). Every other penalized smooth in frmtmb ends up multiplied byZ, which is right fors(x)and useless for a body that needs the spline’s value inside an expression-
exp(amp) * ps(age + shift)- because the argument is itself a function of parameters and there is no fixedZto build. The second-difference penalty is eigensplit exactly asmgcv::smooth2random()splitss(): the null space joins the fixed coefficients and the range space becomes one random-effect block with a single variance intheta, so smoothness is estimated jointly with every other variance component.
-
- The
ps()basis is evaluated ON THE TAPE, as divided differences of truncated powers spelled branch free with0.5 * (e + abs(e)). RTMB refusespmax()on an advector and exports noCondExp, so the recursive Cox-de Boor form is unavailable and this is the only construction there is. It agrees withsplines::splineDesign()to 3.1e-13 absolute atk = 12and 1.8e-11 atk = 40, with an AD input giving the same values bit for bit and the taped derivative matchingsplineDesign(derivs = 1)to 4.5e-13.kabove 50 is refused, because a divided difference cancels terms of order(range / spacing)^degree. -
ps()refusesREML = TRUE,quadrature = TRUE,frmtmb_control(profile = TRUE)and multivariate models by name, and reports at fit end how many rows the fitted transformation pushed outside its frozen knot span, where the basis is exactly zero. - FIX a nonlinear body no longer loses a variable that appears only inside a call in FUNCTION POSITION.
all.vars(quote(a * curry(tv)(zv)))isc("a", "zv"): it treats the whole function-position subtree as the callee and drops it, arguments included. Core collected a body’s data variables that way, sotvwas never asked ofdataand the body failed on it with R’s own “object not found”, which names neither the argument nor the fault.bf(y ~ a * curry(tv)(zv), a ~ 1, nl = TRUE)now fits, to the same coefficient asbf(y ~ a * curry2(tv, zv), ...). - BEHAVIOR CHANGE, and a fix: a nonlinear parameter whose design has NO columns (
b ~ 0 + (1 | g), a parameter that is purely a random effect) no longer adds a phantom entry to the parameter vector.paste()recycles to its longest argument, so a zero-column design came back from the coefficient-naming lines as the single name"b_". The parameter entered no likelihood and no linear predictor indexed it, and the outer Hessian was singular in exactly that direction, sovcov(),confint()and every standard error came backNaNwith “the model is probably overparameterized”. - What changes for existing code:
betaloses ONE entry per such parameter, so a script with a hand-writtenstart = list(beta = )for a model that has one now fails withstart$beta must have length <n>. Drop the element that stood for the parameter with no fixed design; it was never estimated, andpar_template(fit)names what is left, in order. Fits do not move: the phantom entered no likelihood, so the optimum, the log-likelihood and every coefficient are unchanged, and what was lost before and is recovered now is the standard errors. Two of this package’s own tests carried such a vector and were edited for it. - NEW a family may declare
post$fit_check(fit, resp), run once when a fit finishes.logLik()reads the optimizer’s own value, so a family whose likelihood is floored in some region of the parameter space had nowhere to say so and no extension could gatelogLik()orAIC(). This is the hook that closes it. -
frm_compat()gainsps()as a special andfrm_lp_basisas a method, with the rows for both.
frmtmb 0.51.0
One simplex per monotonic term, matching brms; importance sampling over several random-effect blocks; registration seams that refuse what they cannot describe; a simulator for every family but one, checked against its own density; and three brms comparison tiers that pin what still differs.
- BEHAVIOR CHANGE, and the reason the 0.50.0 entry below withdrew its claim rather than its behavior: a
mo()variable that appears in more than one term now gets one simplex per TERM instead of one per variable.y ~ mo(x) * zfits a shape for the main effect and a second for the interaction, so it gains the simplex’s free coordinates,D - 1of them, for every term beyond the first. Such models change: the shared simplex was a constraint on the fitted surface, so removing it can only raise the log-likelihood, and the coefficients, standard errors, information criteria and predictions all move with it. A formula in which everymo()variable appears once is unaffected. - Where the interaction in such a model is near zero its simplex is barely identified, and maximum likelihood runs it to the boundary. The shape estimates stay finite, but their standard errors and intervals can come back enormous and carry no information: on the brms tier’s own
y ~ mo(inc) * zdata the second simplex reports standard errors in the thousands. Read that as the data not supporting a second shape, and write the additive model. There is no option to restore the sharing. - The v0.18 entry justified the sharing as the “brms convention”. It is not, the claim was withdrawn in 0.50.0 below, and the behavior now follows brms too. brms builds one simplex per special TERM and enumerates those terms in
stats::terms()order, which lists every main effect before any interaction whatever the formula wrote. The frame re-sorts into that order before it numbers the simplexes, so the j-th monotonic term of a predictor is the one brms callssimo_<j>and names in thesimorows of itsget_prior(). The monotonic design columns follow the same order, soy ~ mo(x) * znow reportsmoxahead ofmox:z. What lines up with brms is the ORDER, not thezeta<j>number: simplexes continue the numbering of any parameters the family declared first, so an ordinal orcox()fit spends slot 1 on its thresholds or baseline hazard and its first simplex iszeta2. That numbering is unchanged from earlier versions. - The brms log-density tier pinned the divergence by asserting the structural difference at
y ~ mo(inc) * z. That row now asserts the identity, with the flat Dirichlet admitted once per simplex, and the tier’s exemption list no longer holds amo()entry.
Two seams an extension had to work around: a compatibility rule that named a feature nobody had, and a family that needs either of two addition terms.
frmtmb_register_compat()now REFUSES a rule whose sides the registry cannot resolve, rather than accepting it and matching no pair. A rule that matches no pair is dropped without a word, so the contributing package’s table reads as complete while covering less than it claims, which is the worst thing a compatibility surface can do. It happened: a drift-diffusion extension wrotemixture()where the vocabulary saysmixture, and nameddec()before that term was a feature at all, and lost twowienerrows for a release. The refusal names the rule, the spelling, and the nearest entry in the vocabulary when one is close. Unknown kinds ("kind:fmaily"), unknown groups and statuses outside the declared five are refused the same way, as is a rule naming one feature on BOTH sides, which cannot match either: the resolved table holds unordered pairs of distinct features. Nothing is committed until every rule resolves, so a refused registration leaves the vocabulary as it found it.-
frmtmb_register_compat(expects = )is the one legitimate exception: feature names the rules refer to but the registering package does not supply, because another package does. They are exempt from the check and are NOT added to the vocabulary, so such a rule lies dormant until the package that owns the feature is loaded, which is what makes the rule true.frmtmb.sampleneeds it for itshmm x frm_sampleandlca x frm_samplerules, which it only suggests the owner of; before this, those two rows were silently dropped in every session that did not also loadfrmtmb.latent.The argument declares an expectation, so it is held to one at both ends. A name the vocabulary already holds is refused, because it exempts nothing. A name still unresolved when
frm_compat()is called is REPORTED there, in anunresolvedattribute the print method names under the rows, rather than leaving the rule to match no pair in silence. It cannot honestly become a row: there is no such feature in the session, so there is no pair, and inventing one would put a feature in the table that nothing implements. The report is also what makes a misspelling insideexpectsfindable, which registration cannot do, since the whole point of the argument is that the owner is absent. frmtmb_register_aterm()now contributes the term to the compatibility vocabulary as well, as the feature"<name>()"of kind"aterm". A registered term thatfrm_compat()cannot describe is a gap by construction:frm()would accept a term the table refuses to answer about. A package that also declares the feature infrmtmb_register_compat(features = )meets a no-op rather than a duplicate, and one display name registered under two kinds is an error. A name the vocabulary already holds under another kind is refused, and neither registry keeps anything:s()is a smooth, sofrmtmb_register_aterm("s")would give one spelling two meanings in one formula, and thirteen names (s,t2,mo,ar,ma,mm,mmcand the rest) are taken this way.-
frmtmb_family(required_aterms = )now takes ALTERNATIVES as well as a conjunction. A character vector still names the terms a density needs all of. A list adds the choice: an element of length one is required, and an element of length more than one is a set of spellings, any one of which will do, sorequired_aterms = list(c("dec", "vint1"), "vreal1")reads as one ofdecorvint1, andvreal1. The refusal names every alternative in an unmet group and writes the first into its example formula. A family whose per-row datum arrives under either of two terms had to hand-roll that refusal insidevalid_y(), where it fires after the frame is built and where nothing about the framework can check it.Two smaller changes to the same argument.
required_atermsnow refuses a missing or empty entry,c("vint1", NA), which used to be accepted and could then name no term; andlist()is now accepted as no requirements at all, which the character-vector-only check used to refuse. Neither should reach a family that declares its requirements correctly today. The brms log-density identity tier covers eight more shapes of the plan’s matrix: multivariate responses with
set_rescor(TRUE)and with(1 | ID | g)merged across responses (rows 8 and 9),mi()imputation together withmi(sd)measurement error (row 4), a nonlinear model with a random effect on an nlpar (row 6),s()andt2()(row 11), the Hilbert-spacegp(k =)(row 10),ar(cov = TRUE),cosy()andunstr()residual correlation (row 18), andcar()in its escar and icar forms (row 19). Four divergences over three rows are recorded and asserted rather than skipped: the exactgp()nugget, brms’sar(cov = FALSE)likelihood, and the esicar and bym2 CAR parameterizations. Seedev/brms-likelihood-tests.md.tests/testthat/test-brms-methods.Ris a new opt-in tier that compares what the post-fit methods RETURN against brms, rather than what the objective evaluates to. It builds abrmsfitwhose draws ARE frmtmb’s estimates, by feeding the log-density tier’s translated parameter list to Stan’sFixed_paramalgorithm, so brms’s posterior mean is a point estimate and the comparison is exact rather than toleranced.posterior_epred(),posterior_linpred(),log_lik()per row,fixef(),ranef(),VarCorr(),hypothesis()point estimates and the one-wayconditional_effects()grid and estimate all agree to the last bit across the shapes the translator handles. Where they do not, the difference is recorded and asserted indev/brms-methods-tests.mdrather than tolerated, each with a verdict: 10 are frmtmb defects, 8 are paradigm differences that are right for a maximum-likelihood fit, and 8 are open design choices. The sharpest isconditional_effects()on a zero-inflated fit, which plots the conditional meanexp(eta)instead of the expected response(1 - zi) * exp(eta). On the tier’s own data the plotted curve is 15% above the expected response at the first grid point and 268% above it at the last. The same defect is live on the hurdle families, where the error changes sign along the curve and onlymethod = "predict", which draws responses, reaches the expected response, and neither case needs brms to see:fitted()andpredict(type = "response")on the same fit already disagree with the plot.
The simulators are now tested against the densities they are supposed to match.
New test tier,
tests/testthat/test-simulate-density.R. For every family that declares a simulator it draws through the family’ssimslot on a fixed two-cell design and tests the draws against that same family’s ownlpdf, evaluated numerically: a chi-square goodness of fit over cells whose probabilities are summed or integrated from the density, and the first two moments taken from the same measure. This is the one place in the suite that deliberately closes the loop. Everywhere else a validation reference is written by hand precisely so that it cannot share a convention error with the code under test; the cost of that rule was thatfrm_simulate()had never been checked as a generative process, and a shape/scale swap or a zero-inflation gate read the wrong way round would have cancelled between the simulator and the density without anything noticing. The tier covers the addition terms that change a draw (trials(),trunc(),se()) and the ones that must not (weights(),cens()), the ordinal, categorical, multinomial and mixture families, both entry points, and the refusal a family without a simulator has to raise.tweedie(),compois()andhurdle_poisson()have simulators, sosimulate()andfrm_simulate()work for them, and so doesposterior_predict()through frmtmb.sample. Each states a generative process rather than inverting the density: the Tweedie is drawn as the compound Poisson sum of gamma jumps it is on1 < p < 2, which is also where its point mass at zero comes from; the hurdle Poisson clears its hurdle and then draws a zero-truncated Poisson by inverse transform above the Poisson’s own zero, rather than by a rejection loop that would spin at the small mean these models are used at; and the COM-Poisson solves for the rate that puts the distribution’s mean where it was asked for, then inverts its own defining weights.Only
cox()now declares no simulator, and it says why: inverting a cumulative baseline hazard identified only on the observed time window has no answer beyond the last event. The tier reads the list of families with no simulator off the registry rather than carrying it, skipping the deferred entries whose simulator arrives with the data, so a family that gains or loses one changes the test’s answer instead of drifting away from it.Bug fix:
frm_simulate()refused a family whose simulator is installed by itsfamily_finalize()slot, reporting that the family had no simulator, whilesimulate()on a fit of the same model worked. A family that derives itself from the response is not itself until frame assembly has run, andfrm_simulate()was reading the family as written; it now carries the finalized responses over from the frame, asfrm()already did. This is what madefrmtmb.eam::gddm()unreachable fromfrm_simulate().frm(importance = )now corrects a model with SEVERAL random-effect blocks over one grouping factor, which is what distributional regression produces:(1 | g)inmuand(1 | g)insigma,(1 | g) + (0 + x | g), or a block inmuand one in a distributional parameter. A grouping level’s coefficients from every block are drawn together from one joint proposal, so the blocks stay coupled through the group’s rows the way the likelihood couples them. Blocks over different grouping factors are still refused, and the message names the blocks and the factors: crossed or nested factors make the marginal likelihood one integral over every factor at once, which does not split into the per-group integrals the correction resamples.Validated against a brute-force reference that shares no code with the package. On 12 groups of 10 rows with
(1 | g)inmuand(1 | g)insigma, each group’s marginal likelihood was integrated by two-dimensional Gauss-Hermite quadrature over(b_mu, b_sigma)againstdnorm()directly. The reference is 193.161829; the correction at 2000 draws gives 193.137101, which is 0.61 of its own Monte Carlo standard error of 0.0404, while the Laplace approximation gives 193.396458, or 5.8 of that error. The same comparison under a likelihood no identity fixes, 30 groups of 8 Bernoulli rows over(1 | g) + (0 + x | g)against a two-dimensional quadrature ondbinom()directly, puts the correction 0.26 of its Monte Carlo standard error from the reference at 8000 draws, where the Laplace approximation is 56 of them away.
frmtmb 0.50.0
The Laplace approximation, corrected by importance sampling; and get_prior() answers for one route, whatever is attached.
frm(importance = )replaces the Laplace approximation with an importance-sampling correction of it (Skaug and Fournier 2006, the ADMB-isoption). Each group’s marginal likelihood is re-estimated by drawing from the Laplace Gaussian and reweighting the draws, which is unbiased for the exact integral at every parameter value rather than accurate toO(n^-1)near the optimum. A positive integer is the number of draws per group;0, the default, is off.This is the correction for the case
quadrature = TRUEcannot reach. TMBad’smarginal_gktransform is one-dimensional in its C++, so a block of dimension two returnsNaNand thedimin the spec is never read. The importance correction takes any block dimension. On the package’s probe design, 60 groups of 8 Bernoulli rows with a correlated random intercept and slope, the Laplace negative log-likelihood of 299.302 and intercept of -0.691 become 297.915 and -0.707 at 2000 draws, against GLMMadaptive’s adaptive Gauss-Hermite values of 297.919 and -0.705. The gap is 0.12 of the fit’s own Monte Carlo standard error, and 0.015 of the intercept’s standard error.trunc()andcens()are corrected rather than refused. This is a real difference from quadrature, which must refusetrunc()because its far nodes drive the truncation normalizer to underflow; importance draws stay within a few conditional standard deviations of the mode. Both were measured, not assumed.The fit reports how far it can be trusted.
fit$importancerecords the draw count, the seed, the rounds taken, the Monte Carlo standard error of the corrected log-likelihood, and the effective sample size of every group;print()andsummary()give the summary line. Below an effective sample size of 0.25 of the draw count the fit warns and names the groups. That threshold is placed by measurement, and against the anchor the fit reports from: ordinary designs report 0.93 to 1.00, a proposal displaced by half a log standard deviation falls to 0.009, and a genuinely hard design (forty groups of three binary rows) lands anywhere from 0.08 to 0.62, where firing is the point.The corrected log-likelihood is reported from a proposal frozen at the estimate, not at the round’s starting point. The optimizer minimizes the exact objective plus a Monte Carlo error that is exactly zero at the anchor and grows away from it, so the value it stops at is optimistic by construction and by an amount no draw count removes.
A fit whose iteration diverges refuses rather than reporting a collapsed variance component. An unidentified covariance turns the fixed-point iteration into a wander along a ridge, ending where every importance weight is trivially equal and the effective sample sizes therefore look perfect. The invariant that catches it is that the correction must not make the fit worse than the correction at the Laplace estimates.
logLik(),AIC(),vcov()andconfint()come from the corrected objective through the ordinarysdreport()path;ranef(),fitted()andpredict()work as usual, because the conditional modes come from the Laplace inner solve at the corrected optimum, as they already did underquadrature = TRUE.confint(method = "profile")on a corrected fit profiles the frozen proposal rather than refitting, which is deliberate: refreezing at every grid point would make the profiled objective a different random function at each one and destroy the smoothness the profile’s own interpolation depends on, and it was measured at three times the cost besides. What it does now is check the weights at the bound it returns and warn, naming the parameter and the effective sample size, when the bound lands where the proposal no longer covers the integrand. A covariance parameter can walk far enough to get there; a fixed effect generally does not.Each round now starts its optimizer at the current estimate instead of at the cold start template. The reported numbers are unchanged (the probe gives the same negative log-likelihood and the same coefficients to four decimals) and the fit is about 15 percent faster, but the rounds no longer wander: one unidentified design that used to trip the divergence guard now converges to within 0.013 of its adaptive Gauss-Hermite reference.
frmtmb_control()gainsimportance_seed,importance_roundsandimportance_ess. The draws come from a private random stream, so a fit neither reads nor disturbs the session’s random state, and the same seed gives the same answer to the last bit.Everything outside the correction’s scope is refused by name at the earliest point, with the remedy in the message: more than one random-effect block, a block with no grouping factor, a covariance that correlates the grouping levels, a multi-membership term, a second response,
rescor, a residual correlation term, a family that supplies its own log-likelihood, a nonlinear predictor,cs(),mi(),REML = TRUE,profile = TRUEandquadrature = TRUE.frm_compat("importance")lists the rules.vignette("diagnostics")gains a section on when the Laplace approximation is the problem, what the correction costs, and how to read the effective sample size.Behavior change.
get_prior()gainsroute, and itspriorcolumn no longer depends on which packages are attached. Until now the column was filled from the prior-defaults registry on every call, so the same call on the same fit reported(flat)in every slot with frmtmb alone and brms’s weakly-informative defaults oncefrmtmb.samplewas loaded.route = "fit", the default, reports the defaultsfrm()applies, which are flat, and reads no registry: its table is identical whatever else is loaded.route = "sample"reports the defaultsfrm_sample()applies, and refuses, namingfrmtmb.sample, when no loaded package states them. Readers who want the sampling defaults, which is what brms’sget_prior()describes, must now ask for that route.frm_sample()’s own precedence of call over fit over defaults is unchanged; only whereget_prior()reads from has moved. The returned table gains classfrmtmb_prior_rowsand arouteattribute, andprint()names the route on its first line so that a copied table stays unambiguous.A new opt-in test tier checks that frmtmb’s objective is the same function of the parameters as the Stan program brms generates from the same formula with flat priors. At frmtmb’s estimates the two log densities agree to 1e-6 of the log-likelihood, and brms’s gradient there is zero, which is what catches a parameter-map error rather than an arithmetic one. Random-effect models are compared on the joint density at the conditional modes, since brms has no marginal likelihood. The comparison against brms posterior means is gone: the two estimands differ, so its tolerance was wide enough to hide a real divergence.
tests/testthat/test-brms-likelihood.R, gated byFRMTMB_BRMS_FIT_TESTS=trueand run weekly in its own workflow.That tier found one shape the two packages do not spell the same way, and the reason recorded for it in the v0.18 entry below is wrong.
mo()interactions share their variable’s simplex, which that entry justifies as the “brms convention”. It is not the brms convention: brms builds one simplex per special term, so undery ~ mo(inc) * zbrms gives the interaction its own monotonic shape while frmtmb reuses the main effect’s, and frmtmb’s model has two fewer free parameters. Only that claim is withdrawn here. The behavior itself is unchanged and stays the maintainer’s call.
frmtmb 0.49.1
Internal hygiene and the documentation site. No user-facing behavior changes.
- Every
$read of a parameter template, an estimate list, an aterm container, a random-effect, linear-predictor, autocor or eta-design block, a family object, a simulator context, a distributional- parameter list or the model frame is now a[[read, in core and in all four extensions.$on a list falls back to partial matching, so an absent slot returns a neighbor rather thanNULL; that is what once madepars$breadbeta,ctx$mixreadmix_g, andpar_template$thetareadthetaac. Comparing the two trees with every$normalized to[[shows no collateral edit in any of the swept files.dev/bracket-sweep.mdholds the inventory, separating the containers whose collision was already reachable from those that were only prone. -
VarCorr()no longer reads the residual-autocorrelation parameters where the random-effect covariance parameters belong, on a model with an autocorrelation structure and no random effects. The value was discarded before it reached any output, so nothing printed changes. - A source-scanning test keeps those containers clean. It is keyed on the container’s name rather than on
$, so the user-facing spellings (fit$obj,x$fit,object$draws) are untouched, and it polices core and all four extensions from one list. - The reference sites for frmtmb and its four extensions are now one site: each extension builds into a subdirectory of
docs/, and a reference to a function or a vignette in another of the five packages becomes a link. Build them withdev/build-docs.R. - Three published reference pages have been broken since the sampling surface moved out at 0.47.0. Redirect stubs left by that split shadowed real topics, and
loo.htmlredirected to its own URL; pkgdown will not overwrite a file it did not write, so the real pages were unreachable. All three are regenerated. An orphaned copy of the ODE article, which belongs to the extension’s site, is removed from the core site.
frmtmb 0.49.0
One way to bound a thing, a hardened family API, vignettes divided along the package split, and a shipping case study.
Bounds and the last prior classes
- Hard bounds now speak the prior vocabulary and nothing else. A bound addressed by
nlpar,dparorrespwas previously refused (Unknown parameter(s) in bounds: (Intercept)) because it was keyed by the design-matrix column name of the parameter’s own sub-formula; bounds are now keyed by the parameter-template position, so a bound lands where the distribution beside it lands. -
set_prior()gains brms’s class names for the residual-correlation surfaces:"ar","ma"and"cosy"carry a density on the natural coefficient with the transform’s Jacobian (the composite Levinson-Durbin determinant derived in closed form and pinned against numeric differentiation in the suite),"cortime"takeslkj()on anunstr()time correlation, and"rescor"takeslkj()on a multivariate model’s residual correlation.class = "theta"spans all three covariance components by internal name. A first-orderar,maorcosybound maps exactly; at order two and abovelb/ubare refused with the reason and theclass = "theta"alternative. -
frm(lower =, upper =)andfrm_sample(lower =, upper =)are removed, not aliased: every outer parameter they could reach has aset_prior()class. A stale call fails as an unused argument, andfrm_sample()refuses the names explicitly because its...would otherwise have passed them to the sampler and sampled unbounded in silence. -
get_prior()no longer offers a random-effectthetarow for a model with a residual autocorrelation structure and no random effects (a$partial match of exactly the kind the codebase polices, now bracketed with a regression test).
The family API, hardened
-
frmtmb_family()gainsrequired_aterms(an absent required addition term is refused at assembly instead of reaching the density asNULLand returning a fit with a log-likelihood of zero), and the objective refuses any log-density that returns nothing for a non-empty response.family_finalize(fam, y, aterms)lets a family derive link bounds from the data, and the slot call order it relies on is documented from measurement. - New
frmtmb_register_aterm(): a package adds an addition term of its own, with an arity and a coercion, instead of routing per-row data throughvint(); the unknown-term refusal names both routes. - Custom link objects are validated at family construction; a non-finite
init_dparsvalue through its link warns instead of dropping silently;anova()warns when two models’ fixed effects are non-nested by name and by column space; the compatibility registration seam has its own documented page.
Vignettes and the case study
- The posterior diagnostics and the posterior half of the brms migration live in frmtmb.sample as documents of their own (
vignette("posterior-diagnostics", package = "frmtmb.sample"),vignette("brms-posterior", package = "frmtmb.sample")); core keeps the maximum-likelihood halves and points across. The case-studies wiener section is replaced by a shifted-lognormal custom family cross-checked against the built-in to machine precision, with the drift-diffusion model deferred to frmtmb.eam. - New
habit_prepdata andvignette("habit"): a case study replicating the response-preparation model of Hardwick, Forrence, Krakauer and Haith (2019, Nature Human Behaviour) withnlf()on acategorical()family, the full per-subject sweep run live, and a hierarchical(1 | p | participant)fit replacing 58 separate ones. Data under the MIT license of the authors’ OSF deposit; seeinst/COPYRIGHTS.
frmtmb 0.48.0
The structured-family protocol completes, nonlinear bodies get an AD-safe scope, and the extension API passes its first outsider test.
A nonlinear formula body is now a scope of its own: a function it CALLS resolves in RTMB before the formula environment and the search path, so a bare
pnorm(),qgamma(),plogis(),besselK()ormatrix()in annl = TRUEbody or annlf()is the tape-capable version, with noRTMB::prefix. The prefixed spelling gives the identical fit;stats::pnorm()is the escape hatch; RTMB wins over a same-named function of your own; and only names a body calls are shadowed, so a data frame nameddfstill reads as your object. The shadowed names are the RTMB exports that collide with base or stats and are numerically identical to them (qchisqis excluded because RTMB reimplements it), which keeps the taped and numeric evaluations of one body the same function. One narrow caveat:splinefun()in a body now supports only thefmm,periodicandnaturalmethods; spellstats::splinefun()formonoH.FCorhyman. Extension-author surfaces (family log-densities, structure log-likelihoods) keep explicit qualification.New companion package
frmtmb.eam: the Wiener drift-diffusion family with brms’s parameterization, agreeing with RWiener to better than 1e-12 through a tape-safe two-series blend. It was built against the exported extension API alone, with no core change and no reach into internals, which makes it the API’s first acceptance test from an outsider’s position; its findings ledger drives the next round of interface hardening.hmm()andlca()moved to the companion packagefrmtmb.latent, withhmm_probs(),hmm_viterbi(),lca_probs(),lca_profiles(), their tests and the HMM case study. Install it withremotes::install_github("aforren1/frmtmb", subdir = "extensions/frmtmb.latent"). Nothing about either family changed in the move.mixture(),mixture_mvn()andmixture_probs()stay in core, andmixture_probs()still reports the class probabilities of anlca()fit. The two meet through the family object’s component interface rather than through a class, so neither package has to know about the other.?"frmtmb-extension-api"gains the four accessors an out-of-tree structured family needs and could not reach:frame_block_of()(the read half of a structure’sframe_blockslot),structure_supports_all()(the starting point for a family whose likelihood IS rowwise and which declares a structure only to carry its refusals),mixture_posterior()(one posterior implementation for every family that implements the component interface) andmixture_multimodal_refusals()(the REML and profiling refusals a component-permutation-invariant family owes, issued from one place so that core and an extension cannot word the same fact differently).The core boundary test is no longer a ratchet. Every policed token now maps to an empty list of allowed homes, so no core file may name a structured family or an ODE symbol anywhere at all. Its positive control moved to the extension’s sources, and it is the control alone that skips when
extensions/is not in the tree.Suggests:depmixS4,hmmTMBandpoLCAdropped. They were the reference implementations those two families were checked against, and they travel with them.
frmtmb 0.47.0
The monorepo split: the ODE seam and the sampling surface leave core for companion packages in this repository.
- The sampling surface moved to the companion package
frmtmb.sample:frm_sample(),as_tmbstan(),check_laplace(), and everyfrmtmb_drawsmethod, includinglog_lik(),loo(),posterior_epred()andconditional_effects()on draws. Install it withremotes::install_github("aforren1/frmtmb", subdir = "extensions/frmtmb.sample"). Absent it,frm_sample()is not found, and the fit-side methods that need draws name the package and the call in their refusal. Core keeps every generic it still defines a method for, so oneloo()serves both packages in a session. - Ordinary differential equation dynamics moved to the companion package
frmtmb.ode(frm_ode(), its tests, its vignette). With it go the RTMBode suggestion and theAdditional_repositoriesline: core’s dependencies now resolve entirely from CRAN. - New public interface
?"frmtmb-sampling-api": the internals a sampling extension is written against, as one documented contract. New extension seams:frmtmb_register_frame_check()(a frame check an extension registers at load),frmtmb_register_compat()(compatibility-matrix rows), a prior-defaults registry behindget_prior(), andfrmtmb_ad_overload()(the tape-safe wrapper for user functions, previously internal). -
get_prior()reports the default a slot actually has:(flat)with frmtmb alone, which is whatfrm()does, and the sampling defaults oncefrmtmb.sampleregisters them. -
?frmgains “The Laplace approximation, and how to check it”, naming the in-package remedies and pointing atfrmtmb.sample::check_laplace()for the direct measurement. - Core’s boundary test now asserts zero: no core file names a structured family or an ODE symbol outside the two family homes that remain until they move out in turn.
-
Suggests:codaandbridgesamplingdropped; no core code registers on them any more.
frmtmb 0.46.0
The structured-family protocol completes in-package: hmm and lca ride fam$structure alone, and the extension API is exported.
- New exported
latent_probs()returns the posterior latent-state probabilities of any family that declares them.mixture_probs(),hmm_probs()andlca_probs()are unchanged and now reach it;hmm_viterbi()stays its own decoding pass. - New extension API on one page,
?"frmtmb-extension-api":single_response(),eval_dpars(),fit_extras(),dpar_linpred(),response_mean()andas_frmtmb_family()are the read-only accessors afrmtmb_structure()slot may use, with a stability promise.uni_resp()was internal and is nowsingle_response(). -
frmtmb_structure()gainscheck_fit,unitand acluster_robustcapability flag, andloglikbecomes optional. A structure with nologlikis a capability declaration: the family keeps its rowwise likelihood and carries only the refusals that belong to it, and its capability flags default to TRUE so it names what it refuses instead of enumerating what already works. - No core file branches on
hmm()orlca()any more.fit$frame$hmm_gis gone; a hidden-Markov structure’s data isfit$frame$blocks[[response]]. The compatibility matrix takes both families’ rows from the families themselves through a registration the boundary test polices. - One wording change: each mixture-type family now states the REML and
profile = TRUErefusals in its own name. The message previously listedmixture(),mixture_mvn()andlca()whichever one was fitted; everything after the colon is unchanged. - Fits are bit-identical to 0.45.0, and
frm_compat()resolves every feature pair to the same status.
frmtmb 0.45.0
The structured-family protocol: the first step of the core and extensions split.
- New exported
frmtmb_structure()declares a likelihood that does not factorize over the rows of the data. It carries the non-rowwise log-likelihood, the frame block that likelihood reads, the frame variables andNApolicy it needs, a conditional mean and variance, a structured simulator, and twelve capability flags, each with the message its refusal shows.frmtmb_family()gainsstructure =. A structured family can now be written outside the package;dev/structured-family-protocol.mdis the contract. -
mixture(groups = )andhmm()reach the core through that one slot. The objective’s response loop has one branch for both instead of one each, the model frame carries oneblocksslot, andpredict(),fitted(),residuals(),simulate(),conditional_effects()and the fit-time REML, quadrature and profile gates read capability flags instead of naming a family. Fits are bit-identical to the previous branches, and no user-facing message changed; the only deleted refusals were unreachable behind generic guards, which an existing test asserts. -
fit$frame$mix_gis gone. A group-level mixture’s grouping isfit$frame$blocks[[response]], and a structured simulator readsctx[["block"]]where it readctx[["mix_g"]]. - Internal reorganization toward the split: the prior and bound machinery the fit route reaches moved from
R/interop.RtoR/priors.R, and a new boundary test pins where core still names the structured families and the ODE seam, as a one-way ratchet the remaining protocol steps shrink to empty. No user-visible change.
frmtmb 0.44.0
Sampling a fit gets the default priors, conditional-effects plots get real facets, and parameter names become discoverable.
Default priors on the fit route
-
frm_sample(fit)now applies the same brms 2.23 default priors the formula route applies. Both routes sample a posterior, and a prior-freebrm()call translated onto a fitted model no longer pins the chain at whatever boundary mode maximum likelihood found. On the kidney model ofdev/brms-vignette-audit.md,sd(patient)goes from 3.1e-4 with a 3e-6-wide interval to 0.49 with a [0.04, 1.14] interval, and the largest R-hat from 1.57 to 1.10. This is a behavior change for existing scripts that sampled a fit:prior = "flat"restores the bare likelihood. - Priors on a sampling call now stack, most explicit winning per slot:
prior =on the call, then the fit’s own prior, then the defaults. Previously an explicitprior =on a MAP fit discarded that fit’s prior entirely. - Because the defaults cover every standard deviation and correlation, the non-centered parameterization is now offered on the fit route as well;
reparameterize = FALSEstill asks for the centered chain. -
prior = "flat"is the opt-out on both routes, andfrm_sample(fit, prior = "flat")is no longer an error. On a MAP fit it also removes that fit’s own penalty, which is taped into its objective, and the call names the prior it dropped. A MAP penalty can never be applied twice: every prior-carrying path rebuilds the objective from the bare likelihood and adds the merged priors once, and a regression asserts it in objective values. -
check_laplace()is unchanged in what it measures: it samples the density the fit maximized, with no default priors, centered. It now asks for that explicitly rather than inheritingfrm_sample()’s default.
Faceted conditional-effects plots
-
plot()on aconditional_effects()result with several condition sets draws one faceted page of small multiples with a shared scale, as brms’sfacet_wrap("cond__")does, instead of one full page per condition that overwrote its predecessor on a normal device. The panels use tinyplot when it is installed; without it the fallback is a grid of base-graphics panels, still one page.ncol =is honored; it was previously absorbed by...and dropped. -
plot(ce, points = TRUE)shows in each panel only the observations belonging to that condition, following brms’smake_point_frame(): a factor condition splits the data per panel, while a numeric condition is a reference value that names no observation and keeps every point. One deliberate divergence: brms pins factors theconditionsframe does not mention to their first level, silently discarding the other levels’ observations; frmtmb subsets only on the variables the user passed.
Parameter names, discoverable and usable
- New
par_template(): the parameter vector’s names and starting values, for a fitted model or for a formula and data alone. It prints as a compact component listing, and the object it returns is edited and handed straight back asfrm(start =)orfrm_simulate(newparams =). brms has no counterpart:brm(init =)takes Stan program names and brms initializes at random instead. -
frm(start =)matches by name. A named vector inside a component overrides those entries and leaves the rest at their defaults, with parentheses optional as everywhere a parameter is named. An unnamed vector keeps the positional contract exactly. Mixing the two in one vector, an unknown name, a name given twice, and a non-numeric vector are each refused by name. Note the edge: a full-length named vector previously had its names silently ignored; names are now interpreted, and a vector whose names match nothing is refused with a pointer atunname(). - A
normal(),student_t()orcauchy()prior on a nonlinear parameter now places its starting value at the prior location, which is how brms uses the same information. The insurance-loss growth curve of the brms nonlinear vignette fits from its priors with nostartat all. A message names what was placed;startstill wins. Other parameters keep the starts they had. - Fixed on the way:
autoscale = TRUEsilently recycled a too-short start vector into the pre-fit’s full-length start. It now goes through the same validation every other start does, and refuses.
frmtmb 0.43.0
The prior interface speaks brms, ar1() evaluates in linear time, the brms vignettes are measured call by call, and parallel chains work on Windows.
The prior interface
-
frm(),frm_sample()andfrm_simulate()takeprior =, brms’s spelling. Thepriors =of earlier releases is renamed, not aliased: no ecosystem spells itpriors, and the package is pre-release. A stalefrm(priors =)fails as an unused argument;frm_sample(), whose...would otherwise pass the name to tmbstan and fit with no priors at all, refuses it by name. The fit object’s field isfit$prior, asbrmsfit$prioris. - New
prior(),prior_()andprior_string(), brms’s constructors.prior(normal(5000, 1000), nlpar = "ult")quotes its first argument and reaches the same machineryset_prior()does.tagandcheckare deliberately absent: both are Stan-program concerns. - A
brmspriorobject built by brms itself is translated row by row wherever a prior is accepted, so a specification copied out of a brms script works even when brms is attached and itsprior()masks frmtmb’s. Rows with no faithful frmtmb meaning are refused by name rather than turned into a different density: atag, brms’s mixturetheta(frmtmb’sthetais the raw covariance vector), and the per-dpar classes such assigma, whose density brms puts on the parameter and frmtmb’s nearest spelling puts on the link scale. brms’s own default rows are dropped with a message, because frmtmb chooses its own defaults. -
set_prior()gainsnlparandresp, and its argument order now follows brms’s (prior, class, coef, group, resp, dpar, nlpar, lb, ub). Undernlpar, class"b"covers a nonlinear parameter’s whole coefficient vector, intercept included, because its sub-formula is not centered; that is what makes the brms nonlinear vignette’s insurance-loss priors land.nlparalso narrows classes"sd"and"cor", separating two blocks on one grouping factor thatgroupalone cannot tell apart.get_prior()gains annlparcolumn. - The formula route of
frm_sample()no longer invents a location default for a nonlinear parameter’s intercept. brms leaves those flat, and the response’s median and mad describe nothing about a rate or a shape inside a nonlinear body; the disclosure message names the parameters left flat.
Linear-time ar1()
-
ar1()andhetar1()evaluate their density at linear cost in the block dimension instead of building a densed x dcovariance and factorizing it on the tape. The density, and therefore every fitted value, standard error and prior, is unchanged: the new form agrees with the old computation to 2.3e-12 in value and 2.2e-9 in gradient, gated by a test that keeps the old dense code inline. The speedup is orders of magnitude once blocks are large (milliseconds where the dense form took seconds at d = 800, and tens of seconds at d = 2000);dev/sparsear1-bench.Rmeasures it on the host at hand. - A long time series is now an ordinary
ar1()block. The multivariate stochastic-volatility model of Skaug and Yu (2014) fits at its published 945 time points in seconds, agreeing with the upstream TMB reference to 8.5e-9, with a slightly better optimum than the reference’s own cold start reaches.
The brms vignettes, measured
-
dev/brms-vignettes/translates the brms vignettes call by call, each model down both paths: the maximum-likelihood fit with its post-processing surface, andfrm_sample()with the posterior workflow. Every one of 567 calls carries a label (clean, spelling, behavior, missing, or refusal), anddev/brms-vignette-audit.mdholds the measured scoreboard: about 7 of 10 model calls port unchanged, about 4 of 10 of all calls. README’s port claim now states both figures. - The audit’s cheapest finding is applied: the draws and fit surfaces refuse with named replacements where they previously failed bare.
loo(),waic(),bayes_R2(),LOO()andWAIC()on a maximum-likelihood fit now say to sample first or useAIC();loo(a, b)says comparison belongs toloo_compare();plot()on draws namesmcmc_plot();update()on draws says to update the fit and re-sample;rescor_matrix()on draws names the fit;expose_functions()on a fit gets the same refusal draws already had. -
conditional_effects(method = "predict")now works on a nonlinear predictor. The guard that refused it asked for a standard error thatmethod = "predict"never uses: its band is a quantile of simulated responses. This is what the brms nonlinear vignette’s facet-per-year display calls; the same call is also about a third faster on every model, because the dead standard-error computation is gone.
Parallel chains on Windows
-
frm_sample(cores = )parallelizes chains on Windows instead of falling back to sequential sampling. The old guard claimed socket workers cannot evaluate the RTMB tape; in fact tmbstan rebuilds the tape on each worker from the serialized objective closure, and frmtmb’s generated closures are self-contained, so the rebuild works and the draws are identical to a sequential run at the same seed. The remaining cost is a fixed several seconds of startup per worker, which a message now states; short chains gain nothing, long chains approach a per-chain speedup. One caveat: adevtools::load_all()development namespace cannot be rebuilt on a worker, so parallel chains need the installed package.
frmtmb 0.42.0
The TMB-examples replication audit, and REML’s semantics documented and validated beyond the gaussian case.
The tmb_examples audit
- Audited all 35 models in the upstream
tmb_examplesdirectories of RTMB and adcomp: 13 replicate throughfrm()at 1e-8 or better, including the SPDE Weibull survival model with censoring, orange_big’s nonlinear growth with 5000 latent effects, socatt’s ordinal GLMM, lr_test’smap =restrictions written as plain formulas, and a million-row regression; 3 are expressible but pinned or cost-capped; 19 name the capability that blocks them.dev/tmb-examples-audit.mdcarries the table, and every replicating example is a self-contained regression intest-tmb-examples.Rwith its reference likelihood written inline. - The Kalman-filter node the roadmap held open is closed by measurement rather than opinion: the filter and the Laplace route agree to 1e-13 on the multivariate random walk, both are linear in series length, and
MakeTape()$atomic()collapses the filter to nine tape nodes with no adjoint to derive, so the node is free if ever wanted. What blocks state-space models is the formula grammar. The ranked roadmap that falls out puts a sparsear1()covariance first: the dense block is cubic in its dimension, and it is the one thing keeping the multivariate stochastic-volatility example from fitting at its published size. - Two RTMB traps recorded for nonlinear formula authors: a bare
pnorm()orqgamma()in annlbody resolves lexically to stats (qualify withRTMB::), andVectorize()strips the automatic-differentiation class from its arguments.
REML beyond gaussian
- The
REMLdocumentation now states what the flag computes: exact Patterson-Thompson REML for gaussian models, covariate-dependent sigma included, and for every other family the Laplace-approximated integrated likelihood under a flat prior on the mean coefficients, first-order equivalent to the Cox-Reid adjusted profile likelihood. Distributional-parameter coefficients are deliberately not integrated, matching nlme’s varFunc estimation and the double-GLM literature. - Validated three ways in
test-reml-nongaussian.R: glmmTMB agreement for poisson and bernoulli GLMMs at 1e-5, the Cox-Reid identity verified by hand at the pinned REML optimum, andgls(method = "REML")exactness at 1e-5 for varIdent- and varExp-shaped sigma models. Theanova()design rule is asserted for non-gaussian REML fits too.
frmtmb 0.41.0
The tmbstan wrong-density defense, both argument dialects on the draws surface, and the pkgcheck container as a local pre-push gate.
The tmbstan defense
-
frm_sample()andas_tmbstan()refuse, before any sampling, a tmbstan installation that silently samples the wrong density. tmbstan’s install-time generator patches only the firststd_normal_lpdfplaceholder in the stanc output; stanc 2.39.0 (shipped by StanHeaders 2.39.1, on CRAN 2026-09-02) emits two log-density overloads, and HMC reads the unpatched one, so every chain samples a standard normal instead of the model, likelihood and priors alike. The check is one cached read of the installedmodel.hpp, and the refusal names the defect and the remedies (a prebuilt tmbstan binary, or a rebuild against StanHeaders 2.32.10). Affected in the wild: Linux CRAN installs of tmbstan built against StanHeaders 2.39, and source installs elsewhere; prebuilt binaries are safe.dev/prior-dropping-investigation.mdcarries the proof and the upstream issue text. - Three CI cycles of chain-agreement failures in pkgcheck’s container were this bug, not chain luck: the container reproduces them deterministically, and a one-line patch to tmbstan’s generator makes it reproduce this machine to every printed digit. The gates rationale in the test helpers is corrected accordingly; the ungated correctness assertions are what caught it.
Both dialects on the draws surface
-
posterior_epred(),posterior_linpred(),posterior_predict(),predictive_interval(),predictive_error()andpp_check()gain brms’sre_formulaalongside there.formthey have shipped with. Either spelling alone is the setting, both together is refused naming the pair as one setting, and every default is unchanged. brms’s ownposterior_epred.brmsfit()carries the same two spellings. One consequence for positional calls:re.formnow sits one position later, so a value landing there by position meets the both-set refusal loudly rather than silently changing meaning. - The random-effect switch now takes effect IN-SAMPLE for
posterior_predict(),predictive_interval(),predictive_error()andpp_check()on draws: it used to be consulted only undernewdata, sore_formula = NAon training rows silently kept every random effect. A structured family (hmm, group-level mixtures, residual correlation) refuses the switch by name, since the structured draw is the group-level content it would remove. -
pp_check()on draws previously accepted neither spelling and leakedre.forminto bayesplot as an unknown argument. -
set_rescor()takes brms’srescor;rescor_valuestays as an alias. Newdev/brms-api-diff.md: the formals of all 92 exports shared with brms 2.23.0, diffed and triaged.
The pkgcheck container, locally
-
dev/run-pkgcheck-docker.shanddev/pkgcheck-docker.mdrun rOpenSci’s pkgcheck-action container against the source tree on Windows docker, mirroring CI line for line, with a gates on/off switch for the sampler-agreement tests and a token-scrubbed dependency cache. This is the pre-push gate, and it is the instrument that found the tmbstan defect.
frmtmb 0.40.1
-
conditional_effects()gainsre_formula, in brms’s spelling and with brms’s default:NAdraws the population-level curve,NULLconditions on the grid’s reference group levels (pick them withconditions =), and a one-sided formula keeps the named terms, on the fit method and the draws method alike. Previously the setting was hard-coded internally, so the lme4 spellingre.formdied with a matched-by-multiple-arguments error on a fit and both spellings vanished silently into the dots on draws. A user who reaches forre.formhere is now told which spelling this function takes:conditional_effects()is brms’s function and speaks brms, while the fit surface’spredict()andsimulate()keep lme4’sre.form.band = "profile"exists only for the population-level curve and says so. - The message-uniqueness test failed open on one CI layout that offered an existing-but-empty
../../R; it now requires positive identification of the package source tree and skips otherwise.
frmtmb 0.40.0
Input validation across the export surface after an rOpenSci autotest sweep, the submission materials themselves, and the retirement of hand-maintained counts in favor of an enforced uniqueness property.
Input validation
- Scalar arguments across the export surface now refuse a wrong value by name instead of quietly doing something else.
isTRUE()reads a length-2 logical, a string, an integer andNAall asFALSE, sobf(y ~ x, nl = "yes")used to build a linear model andfrmtmb_control(profile = 1L)used to turn profiling off. Nineteen flags are affected, along with the count, coverage and named-list arguments beside them, acrossfrm(),frmtmb_control(),bf(),mvbf(),set_rescor(),predict(),simulate(),confint(),conditional_effects(),frm_simulate(),frm_bootstrap(),frm_multiple(),diagnose(),vcov_cluster(),ranef(),lca(),cox(),frmtmb_family(),prior_normal(),prior_t()andcheck_custom_family(). Every refusal is exercised bytest-input-validation.R. -
confint(level = )andconditional_effects(prob = )refuse a length-2 coverage. They used to recycle it: oneconfint()table came back with its odd rows at 90 percent and its even rows at 95, with nothing in the output recording it. - A link must be named by a single string:
bernoulli(link = c("log", "x"))returned the string as if it were a link, andlink = 1Lselected the identity link by position. -
frm(start = )refuses anything that is not a fully named list; an unnamed value used to be silently ignored, so the fit ran from the defaults and reported them as the user’s.frm(dry_run = ),frm(data = NULL),frm(control = ),frm(na.action = ),predict(newdata = )andpredict(re.form = )all name their contract instead of failing downstream. - One behavior change worth noting:
ranef(fit, condVar = 1)used to work through a truthyif ()and now refuses; writecondVar = TRUE.
Condition messages: property, not count
- Every condition message template in
R/(stop(),warning()andmessage()alike) is asserted unique bytest-message-uniqueness.R, which parses the sources at test time and fails on the first duplicate. The hand-maintained count and its ledger are gone from the standards prose: a number there was stale the commit after it was written, and the test fails at the moment of drift instead. What the walk certifies is template uniqueness (a reported message resolves to one line of source); the shared validation helpers interpolate the argument name at run time and say so.
rOpenSci submission materials
- An audit of every
@srrstatstag against the code it sits on corrected 30 claims that had drifted over 39 versions, including ten internal functions that had lost their roxygen and a not-applicable standard that was actually met and tested. The Bayesian and Monte Carlo category argument is rewritten around the package’s documented inferential surface, with every citation verified against source. - README gains the required statement of need and an installation section;
codemeta.jsonis regenerated;dev/submission-draft.mdfills the submission template, with the r-universe dependency precedent verified from brms, bayesplot and bridgesampling’s own DESCRIPTION files. -
?frmdocuments whatdataaccepts: a tibble, a data.table and a plain named list all work, a matrix column is a supported model variable, and a list column is not one.
frmtmb 0.39.0
The LKJ correlation prior closes the last default-prior gap against brms and brings correlated blocks into non-centered sampling; population-level prediction becomes exact on models with random smooths; and a mechanical lint pass lands with its refusal policy written down.
Breaking: population-level prediction with random smooths
-
predict(re.form = NA)andconditional_effects()now give a true population-level prediction on models with random smooths. A smooth whose basis is indexed by a grouping factor (bs = "fs", a factorbs = "re", a tensor product with anremargin, classified from the smooth object rather than thebsstring) is dropped along with the(x | g)blocks; population smooths,gp()andhsgp()keep their wiggle. The result matchesmgcv::predict.gam(exclude = )on the factor-smooth term to 3e-9. Previously all smooth blocks were kept, sore.form = NAon such a model returned the per-subject curve. This deliberately follows mgcv rather than brms, whosere_formula = NAkeeps factor-smooth curves; the divergence and the reason are documented in?predict.frmtmb_fit. -
predict(newdata = , re.form = NA)no longer needs the grouping column of a factor-smooth term at all, since the dropped term is never rebuilt. When the column is needed and absent, a named error lands before mgcv’s internal one, and any other missing smooth column is named too. - An unseen level of a factor-smooth term errors by default like any other new grouping level;
allow_new_levels = TRUEpredicts it at the population level for anfsbasis (equal tore.form = NAexactly), and is refused by name for a factorbs = "re"basis, whose design has no zero row to hand a new level. -
conditional_effects()on a fit whose only predictors are matrix columns (scalar-on-function regression) names those columns instead of claiming nothing is plottable, and no longer offers a factor-smooth’s grouping factor as an effect.
The LKJ prior on random-effect correlations
-
prior_lkj(eta)andset_prior("lkj(eta)", class = "cor")put an LKJ prior on a random-effect block’s correlation. Class"cor"addresses a whole block, bygroupexactly as class"sd"does, which is brms’s spelling. The density is the LKJ one carried onto frmtmb’s unconstrained row-normalized Cholesky parameters with the exact Jacobian of that map, so on those parameters it means whatlkj(eta)means on the correlation matrix. It coversus()andgr(cov = )blocks of two or more terms, andcs(),ar1()andhetar1(), whose single bounded correlation takes the LKJ marginal(1 - rho^2)^(eta - 1)with that structure’s own Jacobian.toep()is refused by name: its parameterization is not positive definite everywhere, so it has no correlation matrix to put a density on. - The formula route of
frm_sample()now defaults tolkj(1)on every correlation, which is brms’s own default and the last class where the two disagreed. Seeded draws differ from v0.38 for any model with a correlated random-effect block.priors = "flat"opts out, the fit route is unchanged (it is a likelihood diagnostic and keeps flat priors),frm()andcheck_laplace()are untouched, and the disclosure message gains acorline. - Correlated blocks are now sampled NON-CENTERED on the formula route. The gate did not move: a block is non-centered when every parameter it has carries a prior, and correlations now do.
(x | g),cs(),ar1(),hetar1()andgr(cov = )join(1 | g)and the smooths. - Measured on
sleepstudy (Days | Subject), one chain of 2000, three seeds: the flat-correlation median of 36 min-ESS/s with a 154 divergence storm on one seed becomes 120 (centered) and 116 (non-centered) with no divergences, against ~122 for the matched brms model. The prior is what closed the gap; non-centering the block is a wash on this model.dev/benchmarks.mdhas the table and the distribution-level validation of the density. - A MAP fit takes the same prior:
frm(..., priors = set_prior("lkj(2)", class = "cor"))penalizes the correlation toward zero. Maximum likelihood is unaffected, because the default is a sampling default and never entersfrm().
Housekeeping
- A mechanical lint pass: the three
on.exit()calls gainadd = TRUE(none was clobbering another handler), oneany(duplicated())becomesanyDuplicated(), one literal pattern gainsfixed = TRUE.dev/lint-policy.mdrecords what was swept and what is deliberately not chased, with reasons, ahead of the rOpenSci submission.
frmtmb 0.38.0
Non-centered sampling with the funnel diagnosed as a prior, and four case studies: hidden Markov, functional regression, Wiener diffusion, and circular.
Non-centered sampling
-
frm_sample()gainsreparameterize(defaultTRUE): a random-effect block whose every parameter is a standard deviation carrying a prior is sampled non-centered,b = L(theta) z, and mapped back per draw. The draws matrix is unchanged in every respect (sameb[i]columns, same names, same order), solog_lik(),loo(),posterior_epred(),ranef(),conditional_effects()andhypothesis()cannot tell the routes apart. Seeded formula-route draws differ from v0.37;reparameterize = FALSEreproduces the old chains exactly, and the fit route, and with itcheck_laplace(), is unchanged. - Where each group’s own data say little (the regime the funnel lives in) the gain is decisive: a binary GLMM with 80 groups of 2 observations goes from a min bulk-ESS of 5 to 236. Where groups are informative it is a wash, and it is not offered as a blanket speed-up.
- A block is non-centered only when every parameter it has is a standard deviation carrying a prior, and
frm_sample()names every block it left centered, with the reason. This is measured, not conservatism: under a flat prior the reparameterization hands the chain a flat tail the centered geometry was blocking. - The correlated-slopes sampling deficit against brms is diagnosed, and it was not the centering: the flat prior frmtmb puts on a correlation’s unbounded parameter is
(1 - rho^2)^(-3/2), improper, and a proper prior on that parameter fixes it in the CENTERED parameterization, beating the matched brms throughput. brms is ahead there becauselkj(1)is proper, not because it non-centers. An LKJ default on correlations is the queued follow-up;dev/benchmarks.mdcarries the measurements. -
check_laplace()reports each parameter’s bulk effective sample size and says so when the chain mixed too poorly to judge the approximation, instead of letting a sick chain read as a Laplace problem.
Case studies
-
vignette("case-studies")gains four sections: a two-state gaussian hidden Markov model on an animal track, validated against hmmTMB from generic starts (5.9e-10 in the log likelihood) and depmixS4 (9.7e-8); function-on-scalar regression throughs(t),s(t, by = x)and abs = "fs"factor smooth, plus scalar-on-function regression through a matrix-column linear functional term, both validated againstmgcv::gam()under ML; the Wiener first-passage density written as acustom_family()(RWiener agreement 8.7e-11 over 162 settings), with a data-bounded non-decision time through a link object; and von Mises circular regression with a cyclic-basis mean and distributional kappa, against the closed-form circular MLE to 6 decimals. - Documented along the way: a smooth model’s
logLik()equals mgcv’s ML smoothness-selection score-gam$gcv.ubre(notlogLik.gam);frmtmb_family(links = )accepts a link object, which is how a bounded parameter keeps its support without a tape branch. RWiener joins Suggests, used only by the vignette and its density test.
frmtmb 0.37.0
Leave-one-out cross-validation and the brmsfit method surface on draws, the frm_ode() pharmacometrics tier, influence plots and naming ergonomics, and the root cause of the ODE sampling defect, with patches for upstream.
Leave-one-out cross-validation
-
log_lik()onfrm_sample()draws gives the ndraws x nobs matrix of per-observation log-densities, each row at that draw’s own parameter vector and conditional on its own group-level values, brms’s convention (tmbstan samples the random effects too). Addition terms enter through the objective’s own per-row density code, factored out for exactly this purpose, socens(),trunc(),weights()andtrials()cannot drift from what was fitted. -
loo()andwaic()hand that matrix toloo::loo.matrix()andloo::waic.matrix(), withr_efffromloo::relative_eff()on the chain structure.loo_compare()ranks the criteria and is loo’s own comparison when handed criterion objects.psis()returns the smoothed weights. Validated against brms on a matched model: elpd_loo within 4 percent of one standard error, log_lik column means correlated at 0.9999. -
bayes_R2()implements the residual-based estimator of Gelman, Goodrich, Gabry and Vehtari (2019) on expected-prediction draws, exactly as brms computes it. - These are posterior quantities:
?loodocuments thatfrm_sample(fit)’s flat improper priors leave them unregularized and recommends the formula route orpriors =for model comparison. - A likelihood with no per-observation column refuses rather than inventing one: R-side autocorrelation,
hmm(),mixture(groups =), in-model imputation, and laplace-marginalized draws. Each namesAIC()orfrm_bootstrap()as the replacement.
brmsfit method surface on draws
- Draws objects gain
as.array(),as.mcmc(),as_draws_array/df/list/matrix/rvars(),posterior_summary(),posterior_interval(),predictive_interval(),predictive_error(),ndraws(),nchains(),niterations(),nvariables(),nobs(),formula(),family(),getCall(),ngrps(),coef(),pp_mixture(), and the bayesplot delegationsmcmc_plot(),pairs(),nuts_params(),log_posterior(),rhat(),neff_ratio(). 65 of the 96brmsfitmethods now resolve. -
conditional_effects()on draws evaluates the effect grids once per posterior draw and bands the curves with their own quantiles: noband =ormethod =to choose, nonlinear predictors and nominal per-category displays work without a delta method, and it runs on formula-route draws that have no maximum-likelihood fit behind them.ndrawsthins the draws for a cheaper curve. - The default
effectsofconditional_effects()now include one"a:b"display per fitted interaction, as in brms; a term of order three or more contributes its leading pair, with the remaining variables at reference values untilconditions =pins them.
Living beside brms
- Attaching brms (or rstantools, nlme, lme4, loo, posterior, bridgesampling) after frmtmb no longer breaks dispatch: every frmtmb method on a generic those packages also export is additionally registered into the foreign generic’s own table, so
conditional_effects(),hypothesis(),fixef(),log_lik(),posterior_epred()and forty-odd others resolve regardless of attach order. Verified against the full surface with brms attached. -
bf()is a plain function and IS masked bylibrary(brms); a brms-built formula reachingfrm()now says exactly that and names the two ways out (frmtmb::bf(), or attach brms first).
Tape-safe scope for user code
- An ODE
dynamicsfunction and acustom_family()density no longer need the"c" <- RTMB::ADoverload("c")boilerplate: RTMB’s tape-safec(),[<-anddiag<-are spliced into the function’s own body (unless it already binds them), and nonlinear formula bodies evaluate with the same names in scope. A helper such a function CALLS still needs its own bindings, because lexical scope does not travel into other functions. - The brms methods frmtmb does not have fail with the reason and the replacement instead of “could not find function”:
loo_moment_match(),loo_subsample(),reloo(),kfold(),bridge_sampler(),bayes_factor(),post_prob(),stancode(),standata(),expose_functions(),restructure(), and the deprecated spellingsposterior_samples(),nsamples(),parnames(),LOO(),WAIC(). - Suggests gains loo (the estimators are its own, so
loo::loo_compare()andloo::pareto_k_table()read the results unchanged) and coda (as.mcmc()givesgelman.diag()one component per chain).
Pharmacometrics in frm_ode()
- Time-varying covariates.
frm_ode(tv = )takes dynamics inputs that change WITHIN a group, as a step function of time: the solve splits at each change point and each segment’s dynamics see that segment’s value as an extra parameter. The values may be estimated (a covariate-dependent clearance that changes at a known visit); the change points must be data, andtv_breaknames the column they come from. Last observation carried forward, rxode2’s rule for covariates, and the state is continuous across a change because a covariate moves the derivative, not the state. A gradient through an estimated step rate matches central differences to 1e-11. - Steady-state dosing. An
eventsrow markedss = TRUEwith aniisays its cycle has already settled: every compartment is zeroed, as NONMEM and rxode2 read such a record, and the cycle is repeatedn_sstimes (default 20) before the record’s time. This is an approximation, not a closed form, and it says so: off the tape the last two cycles are compared and a shortfall pastss_tolwarns. Exact against the analytic one-compartment superposition to 2.4e-10. - Reset events.
method = "reset"sets every state tovalue, which at zero is NONMEM’sEVID = 3and rxode2’sevid = 3. A reset beside a dose at one instant isEVID = 4; the reset goes first, so the table’s row order does not decide the answer. -
iiandaddlcolumns oneventswrite a repeated schedule compactly, expanding to exactly the hand-written rows. -
outputmay be a column instead of a scalar: one state per ROW, over one shared solve. A parent and its metabolite in one assay column, or two species stacked long, is one call rather than two, at half the solves and the identical likelihood. - Proportional-plus-additive error is documented in
?frm_odeandvignette("ode")throughnlf(sigma ~ ...)over the ODE mean. No new machinery, just the spelling. -
parmsmay be empty when every dynamics input is time-varying.
Influence plots and naming ergonomics
-
plot()on aninfluence()result draws the Cook’s distances with the top cases labeled and one dfbetas panel per coefficient, each with the conventional 2/sqrt(n) reference band (Belsley, Kuh and Welsch 1980).whichandaskfollowplot()on a fit. Notcar::influencePlot()compatibility, and the man page says why. -
hypothesis()says so, once per call, when a fixed-effect coefficient shadows a natural-scale name: a covariate literally namedsigmahides the residual SD, and a coefficient spelling outsd_<group>__<term>orar1hides its summary. The shadowed quantity is reachable under a leading dot (.sigma,.sd_g__Intercept), whichvariables()lists exactly when the collision exists. -
vignette("inputs")gains a naming-collisions section: which vocabularies can meet, which meetings are refused, and the actual resolution precedence in a nonlinear body (nonlinear parameter, then data column, then distributional parameter). -
frm()andfrm_sample()bounds andconfint(parm =)accept the bare name of an intercept-only nonlinear parameter (laforla_(Intercept)). A nonlinear parameter with several coefficients is refused, naming the full spellings.
The ODE sampling defect, root-caused
- The v0.36 finding that
frm_sample()on anfrm_ode()fit fails at warmup iteration 1 is now fully explained, and it is not frmtmb’s to fix: RTMBode’s solver wrapper lets deSolve failures escape as R errors into Stan’s C++ (a hard error when a rate reaches infinity, and an early return with fewer rows than requested).dev/upstream/carries the analysis, a three-patch series against the RTMB repository (verified: the Lotka-Volterra model samples end to end with 0 divergences on the patched build), reproduction scripts, and issue text ready to file. The same investigation found the more-than-8-states ceiling to be an integer overflow in deSolve’s lsoda workspace formula.
Fixed
-
diagnose()no longer relays “NaNs produced” warnings from the pathological fits it exists to inspect; the NaN standard errors are the finding, reported throughbad_se. - The formula-vs-fit sampler agreement test compares the two taped densities pointwise (deterministic) instead of leaning on platform-dependent chains.
frmtmb 0.36.0
Robustness in three senses (numerical, distributional, inferential), per-parameter nonlinear formulas, one simulator per family, direct sampling, and t2() newdata prediction.
Behavior changes
- Draws objects use parenthesis-free parameter names throughout (
Intercept, not(Intercept)), matching brms and whatvariables()andhypothesis()already spoke. This changessummary(ds),fixef(ds),as_draws(ds)andcheck_laplace()output; fits are unchanged, and both spellings still resolve inhypothesis()andfrm_sample(priors =).
Nonlinear formulas per parameter
-
nlf()declares a nonlinear formula for one parameter, as in brms:bf(y ~ a) + nlf(a ~ exp(b * x)) + lf(b ~ 1), and any parameter can carry one -nlf(sigma ~ a + b * z)is a nonlinear model for the residual SD with a linear mean, whichnl = TRUEcannot spell. Nonlinearity is now a property of one parameter; bodies chain to any depth in dependency order, cycles are refused by name, and the oldnl = TRUEmodels are bit-for-bit unchanged. - A body may read another distributional parameter’s per-row value: the spelling for nlme’s
varPower(form = ~ fitted(.)). A data column still wins over a parameter name, so ported brms bodies keep their meaning.frm_ode()works insidenlf()bodies.
Simulation and sampling
-
simulate(),posterior_predict()andfrm_simulate()now share one simulator per family. Three previously silent wrongs are fixed:mixture(groups =)drew a class per ROW instead of per group inposterior_predict()andfrm_simulate(), and residual autocorrelation draws were independent there.hmm()andmixture_mvn()gained simulators on all three paths;cox()explains its refusal at each entry point. -
frm_sample()accepts a formula and samples without a maximum- likelihood fit first:frm_sample(bf(y ~ x + (1 | g)), data = dd, family = gaussian())tapes the model and starts from random inits. The formula route defaults to brms 2.23’s weakly informative priors (replicated againstbrms::default_prior()exactly, zero-shift rule included), announces every default and every deliberate gap (thresholds, shapes, correlations) in one message, andpriors = "flat"opts out.frm_sample()on a FITTED model is unchanged - flat, likelihood-shaped - which is what keepscheck_laplace()meaningful. -
frm(dry_run = "objective")returns the taped model unfitted.
t2() smooths
-
predict(newdata = )works fort2()smooths, including function-on-function terms andt2(..., by = ). A t2 basis carries a second, prediction-only constraint that the fit does not; smooths are now built withsmoothCon(modCon = 3), which drops it and leaves the design, penalties and log-likelihood bit-identical. The refusal is gone.
Student-t random effects
-
(x | gr(g, dist = "student"))gives a grouping term a Student-t latent instead of a gaussian one, brms’s spelling. An outlying group then costs the variance component far less: with one group of 30 displaced by 10 standard deviations, the gaussian latent SD inflates to 1.85 against a truth of 1 while at(5)latent holds at 1.25, and the intercept’s RMSE falls from 0.298 to 0.180. Correlated anddiag()blocks are one multivariate t with a single mixing variable per level, which is brms’s construction and is verified againstmvtnorm::dmvt()to 1e-10. - The degrees of freedom are FIXED, at
dist_nu(default 5), not estimated. brms estimatesnuunder agamma(2, 0.1)prior, and the prior is carrying it: by maximum likelihood the whole grid from 2.1 to 500 sits inside the 95% profile interval at 20 groups, and joint ML runs to a boundary in 24% to 41% of replicates there. This is the frequentist analogue of brms’s ownprior(constant(3), class = "df"). Compare a few values withlogLik()instead. - The reported quantity is the t’s SCALE, not its standard deviation - which is also what brms’s
sd_<group>__<term>is.VarCorr()stores the scale matrix, tags it withnu, and prints aScalecolumn, a convertedStd.Dev.column (scale * sqrt(nu/(nu-2))) and the fixednu, so the convention is visible instead of silent. -
quadrature = TRUEaccepts scalar t blocks and marginalizes them EXACTLY, agreeing with adaptive Gauss-Hermite quadrature to 1e-6. It is the recommended check: the Laplace approximation over a t latent biases the estimated scale UPWARD, by under 2% of a standard error at 8 observations per group but by a factor of three where a near-null variance component meets two-observation groups.dev/tre-feasibility.mdhas the measurements. -
simulate()andfrm_simulate()draw a multivariate t;predict(allow_new_levels = TRUE)inflates the unseen level’s variance bynu/(nu-2);REML = TRUE,ranef(),confint()andfrm_bootstrap()work unchanged. Refused with their own messages, and documented in?frmtmb-student-reandfrm_compat():gr(cov = )/gr(prec = ),mm(),|ID|keys, and every covariance structure butusanddiag.
Cluster-robust (sandwich) standard errors
-
vcov_cluster(fit, cluster = ~ g, type = "CR0")gives the cluster-robust covariance of the estimates: the inverse observed information of the marginal likelihood sandwiching the outer product of the per-cluster scores.typetakes"CR0","CR1","CR1p"and"CR1S", spelled and defined as inclubSandwich.vcov(fit, cluster = ~ g, type = )is the same thing in thesandwich::vcovCL()spelling. -
cluster_scores(fit, ~ g)returns the per-cluster scores themselves, one row per cluster, so any other sandwich can be built from them. This is whatsandwich::estfun()would return if a marginalized objective had per-observation contributions; it does not, which is why frmtmb still ships noestfun()method. -
confint(),hypothesis()andsummary()takevcov =: a covariance over the whole outer parameter vector, or a function of the fit returning one. A matrix fromvcov_cluster()carriesG - 1reference degrees of freedom, so those methods switch from a normal to atreference automatically. - The estimator is refused, with the reason, wherever the marginal likelihood does not factor over the clustering factor: a random effect whose level spans two clusters (crossed effects,
mm()pooled levels, a global smooth,gp(),car(),spde()), a group-level mixture whose groups span clusters,autocor(),hmm(),rescor = TRUE,mi()/me(),REML = TRUE,frmtmb_control(profile = TRUE),quadrature = TRUE, and any fit made with priors.frm_bootstrap()is the documented fallback. -
"CR2"/"CR3"are refused rather than approximated: the Bell-McCaffrey family is defined through the hat matrix of a linear or GLS model, which a Laplace-marginal likelihood with a nonlinear link does not have.
Numerical robustness of the family log-densities
The log-densities now evaluate from the LINEAR-PREDICTOR scale wherever an exact form exists, instead of undoing the link and recomputing. An inverse link saturates - plogis(40) is exactly 1 in double precision, and 1 - exp(-exp(4)) is too - so a density written over 1 - mu returned -Inf with an unusable gradient in a region the linear predictor describes perfectly well. A fit that converges never visits it; an optimizer step that overshoots, a separated predictor, a wide quadrature node and a frm_sample() tail draw all do.
-
bernoulli(),binomial()andzero_inflated_binomial()useRTMB::dbinom_robust(), the log-odds parameterization glmmTMB fits with. They were wrong in the second decimal at a linear predictor of 30 and gave-Infat 40; they are now exact over the whole line. With theclogloglink the old form had no gradient past a SINGLE-DIGIT linear predictor. -
negbinomial(),nbinom1()andgeometric()useRTMB::dnbinom_robust(), which takeslog(mu)and never formsmu / var. The old form gaveNaNat a linear predictor of -40. - Every
ziandhugate, andasym_laplace()’squantile, run in log space, so a separated zero-inflation predictor stays differentiable. -
Beta(),zero_inflated_beta()andbeta_binomial()take both shapes off the log-odds, so the second one no longer collapses to zero. -
hurdle_poisson()’s zero-truncation normalizerlog(1 - exp(-mu))usesexpm1, which keeps it at a tiny mean. -
cumulative(),sratio()andcratio()with the logit link compute their CDF differences in log space.acat(),categorical()andmultinomial()accumulate their denominators withlogspace_add()instead of summingexp(), which used to overflow at a linear predictor of 709 / (K - 1). - Values and gradients at ordinary linear predictors are unchanged to 1e-15, and every fitted model in the regression set lands on the same estimates (largest coefficient change 9.5e-9, largest log-likelihood change 4e-12).
cumulative(probit)keeps the plain CDF difference, which has no exact log-space form; it is fragile past a linear predictor of 8, so prefer the logit link. New filetests/testthat/test-numerical-robustness.Rpins the behavior.
New family
-
huber()fits Huber’s least-favorable distribution: gaussian withinkresidual standard deviations ofmuand Laplace outside, so an outlier pulls on the fit with bounded influence. It is a proper normalized density, so this is ordinary maximum likelihood andlogLik()andAIC()mean what they say. Huber’s tuning constant is a fixed argument,huber(k = 1.345), not an estimated parameter, matching howMASS::rlm()treats it. Point estimates trackMASS::rlm(psi = psi.huber); the remaining gap is the scale (rlm()fixes it at a MAD-type estimate,huber()estimatessigmaby ML), and holdingsigmathere reproducesrlm()to 1e-5. Theasym_laplace()working-likelihood caveat applies: Wald standard errors under a misspecified error distribution are not calibrated, so usefrm_bootstrap().
frmtmb 0.35.0
Hidden Markov models, latent class analysis, multi-membership random effects, three new families (nominal, circular, proportional hazards), profile and bootstrap effect bands, and a brms-portability batch measured against the brms vignettes themselves.
Behavior changes
-
frm()defaults tofamily = gaussian()when neither thefamilyargument nor a+attachment supplies one, the brms/lme4/glmmTMB convention.frm(y ~ x, data = d)was an error and is now a linear model. Thefamilyargument overrides a family attached to a univariatebf()and fills only the empty responses of anmvbf(). - Documentation and examples now lead with the separate-family spelling
frm(bf(y ~ x), family = gaussian(), data = d); the+attachment stays valid and documented as the alternative.
Portability (measured against the brms vignettes)
- A scorecard audit ports every model call of the brms vignettes through the mechanical
brmtofrmtransform (dev/brms-vignette-port.md): with this release’s fixes the measured tally is 30 of 42 model calls running mechanically, the remaining spelling changes documented in the porting guide. -
hypothesis()accepts brms’s directional form ("a > b") with one-sided p-values and bounds across every method, plus theclass=/group=naming shorthand.update()speaksformula./newdataand dotted deltas (. ~ . + z).lf()composes dpar formulas ontobf(). One formula may name several parameters (a + b ~ 1). A bare constructor (family = cumulative) works. - One parameter-addressing vocabulary across
hypothesis(),profile(),confint(parm =)and bounds: parenthesized and paren-stripped spellings are interchangeable, and one-to-one natural-scale names (sd_g__x,ar1) reach their internal parameter with the scale stated. Ambiguous natural-scale names are refused with the alternatives named.
Multi-membership and effect bands
- Multi-membership random effects:
(x | mm(g1, g2, weights =, scale =))andmmc()member-specific covariates, following brms; the pooled-level Z construction matchesbrms::make_standata()exactly, and every post-fit method reads the block unchanged. New-level prediction variance distinguishes a shared new level (one draw, weights add) from distinct new levels (independent draws). -
conditional_effects()gainsband = c("wald", "profile", "boot"): likelihood-ratio bands inverted per grid point, and pointwise percentile bands from one shared parametric bootstrap (reused across effects and verified against the grid it was run over). Effect discovery now works on nonlinear andmo()/mi()fits, andfrm_bootstrap()works on ordinal fits (previously all NA).
New families
categorical()fits a multinomial logit to an unordered factor, brms’s spelling of the likelihoodmultinomial(K)already carried on a count-matrix response. The first level is the reference and each remaining level gets its own linear predictor namedmu<Level>, as in brms; the main formula applies to all of them unless a dpar formula overrides one, sobf(y ~ x, mustout ~ w)gives one category its own predictor.fitted()andpredict(type = "response")return then x Kmatrix of category probabilities with the response’s own levels as column names, the same convention the ordinal families follow, andsimulate()draws factor levels. Validated againstnnet::multinom(4e-10 in the log-likelihood) and againstmultinomial(K)on the one-hot response (1e-8). The categories are read off the data byfrm()before the formula is parsed;categorical(levels =)orcategorical(K =)states them for the paths that have no data.residuals()is refused: a nominal response has no scale for one.von_mises(link = "tan_half")for a circular response in(-pi, pi], with the mean directionmuthrough the tan-half link and the concentrationkappathrough a log link - brms’s parameterization. The normalizing constant’slog I0(kappa)is differentiated exactly by RTMB’s ownbesselImethod, so nothing is a series approximation.kappa ~ xdistributional models work, and the simulator is a vectorized Best-Fisher sampler that varies both parameters per row. Validated against a hand-rolled RTMB likelihood (1e-6) andcircular::mle.vonmises().cox()fits proportional hazards with brms’s flexible baseline: an M-spline baseline hazard over a simplex of weights, with the cumulative baseline hazard the I-spline integral of the same basis (df = 5cubic splines with an intercept, brms’sbhaz()default). Censoring runs through the ordinarycens()addition term - an event contributes the density and a censored row the survivor function - so right, left, and interval censoring all work, and frailty models come free through the Laplace approximation:time | cens(c) ~ x + (1 | g). Newcox_baseline()returns the fitted baseline weights. Validated exactly against a hand-rolled M-spline PH likelihood (1e-6 in the log-likelihood, 1e-4 in the coefficients) and againstsurvival::coxph()to 2e-2 on the log hazard ratio, which is the honest claim:coxph()leaves the baseline fully nonparametric.fitted(),predict(type = "response")andsimulate()are refused - a survival time has no mean the censored rows identify, and brms refuses the same question. Unpenalized maximum likelihood often puts a baseline weight on the simplex boundary, which the optimizer reports as singular convergence even at the optimum;?frmtmb-familiesexplains it and loweringdfis the remedy.New
tan_halflink, forvon_mises().
Internal
A family may declare
aterm_data(y, aterms), family-level data that no addition term supplies, built once at frame assembly from the validated response.cox()’s spline bases use it.A family’s
lcdf()may take the family-level extra parameters as a fourth argument, which is what lets a survival family’s survivor function reach its baseline. The three-argument contract is unchanged for every other family.
Hidden Markov models as a first-class family.
New
-
hmm(K, family, time =, group =, init =, trans =)fits aK-state hidden Markov model. The response at each time point comes from one ofKstate-dependent copies offamily, and the unobserved state follows a first-order Markov chain alongtimewithingroup. The state sequence is summed out EXACTLY by the forward algorithm, evaluated on the same AD tape as everything else, so nothing about the Laplace machinery changes: a random effect in a state’s linear predictor is integrated outside the exact state sum. Each of the wrapped family’s parameters is copied per state and suffixed (mu1,sigma1,mu2, …) with the full formula grammar, random effects included, exactly asmixture()does. Transition probabilities are a row-wise multinomial logit namedtr{i}{j}with state 1 the reference cell of every row, andtrans = ~xgives every cell a predictor at once. Validated againstdepmixS4(logLik to 1e-8 or better and every parameter to five decimals, on gaussian, poisson and categorical emissions with and without transition covariates) and againsthmmTMB(1e-12 on the stationary fixed-effect model). -
init = "stationary"(the default) solves the chain’s stationary distribution on the tape and costs no parameters;"estimated"addsK - 1free logits,"uniform"fixes them. Stationary is refused when a transition cell carries a predictor, because a time-varying chain has no single stationary distribution. -
hmm_probs()returns the smoothed state occupanciesP(S_t = k | y)from a forward-backward pass - themixture_probs()analog - andhmm_viterbi()the maximum-a-posteriori state path.fitted(),predict(type = "response")and the response and pearson residuals all route throughhmm_probs(), so they report the occupancy-weighted meansum_k P(S_t = k | y) mu_k(x_t)rather than any single state’s.simulate()walks the chain forward per sequence and then emits, so DHARMa andpp_check()see the fitted persistence. - A missing response is a time point the chain passes through without emitting:
hmm()keeps the row and masks its emission instead of lettingna.actiondrop it, which would shorten the chain and bias the transition matrix.nobs()counts every row;fitted()andresiduals()areNAthere whilehmm_probs()is not.
Refusals
-
REML,quadrature = TRUEandfrmtmb_control(profile = TRUE)are refused on anhmm()fit, each with the reason. REML in particular used to run and produce a partial restricted likelihood matching no standard definition. -
weights(),cens(),trunc(),se()andmi()on the response, multivariate models andrescor,residuals(type = "osa"),residuals(type = "deviance"),predict(se.fit = TRUE)andpredict(newdata =)on the response scale, andconditional_effects()are refused, all naming the per-sequence likelihood as the reason. - A grouping in which every sequence has length 1 is refused: the transition parameters are then flat directions the reported
dfwould still count, and the model is amixture(). Holding every transition dpar at a constant lifts the refusal, and the fit then reproducesmixture()to 1e-12. - A start with every state’s location predictor at the same value warns: it sits on the label-symmetry axis, where the optimizer cannot separate the states. The default quantile-spread starts never do this.
Known limits
- Multimodality is real and no convergence diagnostic reports it. On a random-effect model the default cold start has been measured converging 8.1 log-likelihood units below the global optimum with
convergence == 0,max|grad| == 3.5e-4and a positive-definite Hessian. Compare starts before reporting. - With random effects the Laplace approximation is genuinely approximate even for a gaussian response, because the integrand is a mixture over state sequences: the measured bias is -0.126 in the log-likelihood (8.9e-5 relative) and 4.4e-4 in the parameters against adaptive quadrature.
?hmmsays so. - The tape build grows slightly faster than linearly in the number of rows: about 1.9 s at 20 000 rows, against milliseconds to evaluate it. Below 5 000 rows nothing is noticeable.
New
- Latent class analysis, the poLCA measurement model, as the family
lca(K). The response is a matrix of polytomous item codes, one row per subject and one column per item; the items are conditionally independent given a subject’s latent class, and each class carries its own item-response profile. Class membership is thetheta1 ... theta{K-1}dpars with full linear predictors, so a covariate on the model formula gives poLCA’s latent class regression for free, withfixef(),confint(),hypothesis(),set_prior()andfrm_sample()on the gating coefficients. Items may have different numbers of categories;ncatdeclares them, and by default they are inferred as the largest observed code per item, as poLCA does. The item profiles are family extra parameters, one vectorpi<j>per item holding itsK * (C_j - 1)reference-category logits, so they appear per item insummary()and aspi<j>_<i>inconfint(). Validated against poLCA on its own shipped data: the carcinoma 3-class model agrees to 4.2e-8 in log-likelihood, 2.8e-8 in item profiles, 6.2e-9 in class sizes and 9.9e-8 in posterior membership, and the election latent class regression agrees to 1.1e-7 in log-likelihood, 1.1e-7 in item profiles and 9.9e-7 in gating coefficients. A hand-rolledoptim()reference on simulated data agrees to 1.5e-9, and a one-item fit reaches the saturated single-categorical likelihood to 5.6e-11. -
lca_profiles()returns the class-conditional item-response probability tables (poLCA’sprobs) with the estimated class sizes attached, and prints them.lca_probs()returns posterior class membership per subject (poLCA’sposterior) with the relative entropy of the classification attached; it ismixture_probs()under an LCA-specific name and check. -
lca(na.rm = FALSE)keeps subjects with missing items and masks each missing item out of that subject’s likelihood, poLCA’sna.rm = FALSEbehavior. The default drops incomplete subjects through the usualna.action, which is poLCA’s default.
Notes
-
lca()starting values are deterministic: subjects are scored by the mean of their item codes rescaled to[0, 1], cut intoKequal-count slices, and each slice’s smoothed category proportions seed one class. Class 1 is the low-score end, so a data set always gets the same labeling. Multimodality is unchanged;?lcashows the perturbed-startloop that replaces poLCA’snrep. -
lca()v1 refuses random effects, smooths andgp()anywhere in the model (that is the growth-mixture shape, whichmixture(..., groups = ~g)fits),REML,profile = TRUE,quadrature, every addition term,mvbf(), andresiduals(type = "osa").fitted(),residuals()andpredict(type = "response")are refused because a matrix of nominal item codes has no mean;predict()returns the gating linear predictor.
frmtmb 0.34.0
Within-group residual correlation (R-side effects), quantile regression completions, and plot conveniences.
New
- Within-group residual correlation, brms’s R-side autocorrelation, as formula terms
ar(),ma(),arma(),cosy()andunstr()forgaussian()andstudent(). The residuals of a group become one multivariate draw,y_g ~ N(mu_g, D R D), with asigma ~ xdistributional model entering through the diagonal. The ARMA autocorrelation function is exact and orders above one are supported (brms caps its covariance form at one). Validated againstnlme::glsandnlme::lmeunder ML and REML to 1e-9 or better across all five structures, on balanced and ragged data, and with random effects alongside the correlated residual. Newautocor_matrix()returns the fitted correlation matrix; the parameters appear insummary(),confint(),confint_varcorr()andhypothesis()under brms’s names. See?frmtmb-autocorfor the deliberate divergences:sigmais the MARGINAL residual SD (brms uses the innovation SD for ar/ma/arma; the migration vignette gives the conversion), lags count gaps in the global time-level set (nlme semantics), and brms’s defaultcov = FALSEresidual-regression form is refused rather than reinterpreted.simulate()draws correlated residuals, so DHARMa andpp_check()see the fitted autocorrelation. Combinations that stop the likelihood factorizing over rows (weights, cens/trunc, se, mi, rescor, mixtures, quadrature, nl, OSA residuals, other families) are refused with the alternative named. -
zero_inflated_asym_laplace()(brms spelling): a point mass at zero mixed with the asymmetric Laplace, for zero-inflated quantile regression. A new “Quantile regression inference” section on the families page states plainly that ALD-based Wald intervals are not calibrated (a property shared with brms) and points atfrm_bootstrap(). -
plot(conditional_effects(fit), points = TRUE)overlays the raw observations, the brms argument, previously ignored. Points are drawn under the bands; displays where raw observations are not meaningful (per-category ordinal, non-mean dpars, matrix responses) say so instead of failing silently. - Hidden Markov models: a feasibility study (dev/hmm-feasibility.md) establishes that the forward algorithm tapes in RTMB, composes with the Laplace approximation over random effects, and is expressible today through
custom_family()withvint()payloads, validated against depmixS4 and hmmTMB. No user-facing grammar yet; the design for a first-classhmm()family is recorded.
frmtmb 0.33.0
Two residue fixes, a documentation dark mode, and CI repairs.
Behavior changes
-
posterior_epred()on an ordinal fit returns adraws x observations x categoriesarray withdimnameslist(NULL, observation names, category levels), replacing the flatteneddraws x (n * K)matrix of v0.31/v0.32. This is brms’s documented shape (an S x N x C array for categorical and ordinal models). The array is the old matrix reshaped:matrix(as.vector(ep), nrow = ndraws)recovers the previous value and column order exactly, andep[, , "high"]orep[k, , ]replace the old naming recipe. Scalar-response families,posterior_predict(), andposterior_linpred()are unchanged. A documentation error is also corrected:multinomial()response-scale predictions are a vector, not a category matrix, and the docs no longer claim otherwise.
New
- A
bf(nl = TRUE)body can name an object of its formula environment that could never be a column ofdata(a data.frame, a list, an environment, a formula), sofrm_ode(..., events = doses)takes a dosing table by name instead of needing an inlinedata.frame()or a wrapper function. The boundary is exactly whatstats::model.frame()refuses: vectors, factors, and matrices are legal model-frame variables and still resolve through the model frame, and a column ofdatastill wins over a same-named object. A body that fails to evaluate reports which names were resolved outside the data. - The documentation site has a light/dark/auto theme toggle.
frmtmb 0.32.0
The merged-|ID| Kronecker path, the ordinal prediction surface, ODE dosing, and vignette figures.
New
-
frm_ode()gainsevents, a per-dataset dosing table (time,value,state, optionalgroup,methodof add/replace/ multiply,durationfor constant-rate infusions) supporting repeated doses, andevent_scale, an estimated multiplier on every dose amount, so bioavailability is an ordinary nonlinear parameter with covariates and random effects. Dosing splits the solve at the event times instead of using deSolve’seventsargument: deSolve events do not differentiate correctly through RTMBode’s augmented system (replaceandmultiplymeasured 42 and 59 percent relative gradient error; reported upstream). A branch on time inside a derivative function is refused by RTMB, and anapproxfun()forcing table inside one is silently wrong; both are documented.vignette("ode")gains a repeated-dosing section. - Terms sharing an
|ID|key whose grouping isgr(g, cov = A)orgr(g, prec = Q)now merge into one Kronecker block instead of being refused:mvbf(bf(y1 ~ (1 | q | gr(id, cov = A))), bf(y2 ~ (1 | q | gr(id, cov = A))))is the same joint density as the long format(0 + trait | gr(id, cov = A)), verified to 7e-10 on the log likelihood under ML and REML and on both the covariance and precision sides. The compatibility rows move from refused to conditional. - Vignettes carry figures where a picture beats a table: a forest plot for the meta-analysis, the monotonic step shape, the location-scale band with the mgcv overlay, growth-mixture trajectories by recovered class, the sleepstudy small multiple, and concentration-time curves in the ODE vignette. Figures use the suggested tinyplot package and are skipped when it is not installed.
Behavior changes
-
fitted()on an ordinal family returns the n x K matrix of category probabilities (the brms convention), exactly aspredict(type = "response")does; it previously returned the latent linear predictor. The invariantpredict(type = "response") == fitted()now holds for every family with no documented exception.predict(type = "link")keeps the latent predictor. -
residuals(type = "response")andtype = "pearson"on an ordinal fit score the categories by their codes:y - sum_k k * P(y = k), standardized by that distribution’s own sd."response"previously returned the observed code minus the latent predictor without saying so, and"pearson"errored. - One
|ID|label spread over more than one grouping specification is now an error. Previously the merge key included the deparsed grouping call, so(1 | q | g)next to(1 | q | gr(g, cov = A)))landed in separate blocks and silently did not correlate at all, which is not what the shared label asks.|ID|-linkedgr()terms whose matrices resolve to different objects are refused at frame assembly, comparing the resolved matrices.
Corrected behavior
-
emmeans()on an ordinal fit no longer fails with “Non-conformable elements in reference grid”; the basis is built by column name (marginal means stay on the latent scale, the emmeans convention forclm-like models).pp_check()anddharma_residuals()work on ordinal fits (simulated ordered factors are compared on their integer codes; DHARMa’s rank transform runs on simulated categories withintegerResponse = TRUE).plot()works on an ordinal fit. -
conditional_effects()on an ordinal fit draws one probability curve per response category (acats__column), with delta-method standard errors over the joint covariance of coefficients, thresholds, andcs()terms, and bands on the logit scale so they cannot leave [0, 1].method = "predict"is refused there;dpar = "mu"gives the latent display.
frmtmb 0.31.0
ODE models, an exotic-models case-study vignette, and the defect wave the vignette work exposed.
New
-
frm_ode()fits ordinary differential equation models inside abf(nl = TRUE)body. Dynamics parameters and initial states are ordinary nonlinear parameters, so they take fixed effects, random effects and covariates, and the Laplace approximation is exact through the solver’s adjoint. It solves one small system per group and scatters the solution back into row order; ragged designs, unsorted rows, repeated times and an observation att0all work. Needs the optional RTMBode package (not on CRAN;Additional_repositoriesnow names https://kaskr.r-universe.dev), and every code path, test, example and vignette chunk degrades cleanly without it. Newvignette("ode")works a population pharmacokinetic model ondatasets::Theoph. -
frm_ode()refuses the ways an ODE model goes quietly wrong: a dynamics parameter that varies inside a solve group is rejected by name at frame assembly; fixed-step integrators such asrk4are refused because they return a different likelihood, not a noisier one; a system near the Laplace ceiling of about eight states warns. A failed solve is reported, not absorbed: on numeric evaluation paths (predict(),simulate(),residuals(), and a body with no estimated dynamics inputs) the group’s rows get a penalty value and a warning naming the group, andfrm_ode_failures()reads the record back. During fitting most failures cannot be detected at all (RTMBode returns an AD object), so they surface as the optimizer’s NA/NaN gradient; the help page says which is which. - New
vignette("case-studies"): eight worked models from the showcase literature of brms, MCMCglmm, metafor and mgcv. The animal model and its multi-trait form, phylogenetic regression, random-effects meta-analysis and meta-regression, monotonic ordinal predictors, location-scale regression, growth mixtures, measurement error, and sequential ordinal models with category-specific effects. Every section cross-checks its fit against a reference package or a closed form, the multi-trait section proves its pedigree matters with an identity refit, andtests/testthat/test-case-studies.Rpins the agreements. Suggests gains ape and metafor for the cross-checks, and data.table and tibble for tabular-input tests.
Breaking
- A
gr(cov = )orgr(prec = )term whose|ID|key is shared with another term is refused at parse time. The cross-formula merge hardcoded an unstructured covariance and silently discarded the matrix, so earlier fits of that construct ignored the structure they named. The error points at the supported long format,(0 + trait | gr(id, cov = A)), which takes the verified Kronecker path. A lonegr()term with an unshared key keeps its structure and still fits.
Corrected behavior
-
hypothesis()andvariables()exposesd_<group>__<term>andcor_names forgr(cov = ),gr(prec = )andequalto()blocks, so heritability-as-ICC is one line on an animal model. Smooth,gp(),car()andspde()blocks stay excluded;?hypothesissays which and why. -
vcov(full = TRUE)works under REML andprofile = TRUE, taking the outer block from the joint precision instead of warning and returning only the fixed effects. Its rows areconfint()’s rows. - The observation-level-random-effect check no longer fires on
gr(cov = ),gr(prec = )orequalto()blocks with one row per level: a fixed relationship matrix identifies the two variances, which is exactly the animal model with one record per individual. -
predict(type = "response")on the four ordinal families returns an n x K matrix of category probabilities named by the response’s levels, honoringcs()on newdata;se.fitthere is refused rather than faked. marginaleffects gets the categoricalgroupconvention (also fixingmultinomial()row alignment),posterior_linpred(transform = TRUE)stays on the latent dpar as documented, andposterior_epred()’s flattened categorical shape is named and documented. -
anova()anddrop1()report NA instead of< 2.2e-16 ***when the compared models have equal df. -
confint_varcorr()no longer reports a zero-width interval at its internal clamp for a correlation estimated at plus or minus 1 or an sd at zero; those rows keep the estimate, get NA bounds, and warn.VarCorr()andranef()handle two blocks sharing a term label (one printed twice, the other silently dropped a block). - A
Date,POSIXctordifftimepredictor is reported at frame assembly with its unit and 1970-01-01 origin plus the centering fix; the epoch-scale magnitude can stop the optimizer converging. Responses and grouping variables are exempt (srr G2.5, G2.9). - A nonlinear formula body that fails to evaluate reports which of its names were resolved to functions instead of found in
data, so a misspelled column sharing a name with a base function (t,c) is named rather than surfacing as a coercion error. - srr completeness:
srr_stats_pre_submit()reports no missing standards (seven were untagged; six were already satisfied and are now tagged with pinning tests, G2.9 is satisfied by the datetime reporting above).
frmtmb 0.30.0
data2, ODE feasibility, and the remaining rOpenSci runway.
New
-
frm(..., data2 = list()), the brms spelling for structural objects: the matrices ofgr(cov =)/gr(prec =),equalto(),car()andspde()resolve fromdata2before the data and the formula environment (compound expressions overdata2objects work, a documented permissive divergence from brms).data2is stored on the fit, sosaveRDS()carries the matrices andrefit()/update()/drop1()/influence()/frm_multiple()re-assemble in a session where the calling environment is gone. - Population pharmacokinetic models work today through
nl = TRUEbodies callingRTMBode::ode()per subject (RTMBode installs from kaskr.r-universe.dev): the Laplace approximation is exact through the adjoint solver, and a one-compartment population fit agrees with nlmixr2’s FOCEi to three decimals with no compiler. The feasibility study, the sharp edges (never stack subjects into one system; adaptive integrators only), and the plannedfrm_ode()helper are recorded in dev/ode-feasibility.md.
rOpenSci runway
- Every exported help page has executing examples (35 pages added; the as-cran check runs them all in ~14s).
- Every internal function (288) is documented with its contract under
@noRd; a new “Inputs and preprocessing” vignette covers the formula-to-design pipeline, accepted predictor classes, argument types, attribute survival, terminology, and reproducible runtime claims. srr standards: 105 of 116 tagged in code, docs, and tests; the other 11 documented as not applicable. - Nine standards tests added (degenerate inputs, noise invariance, exact-fit behavior, row-name retention, runtime scaling).
- CI gained the rOpenSci
pkgcheckaction (the editors’ submission checks, srr included) and a coverage workflow.
Corrected behavior
- Rows dropped by
na.actionare reported with a count (amessage(), sosuppressMessages()silences it); fitting zero rows says so instead of blaming missing values. - Eleven duplicated error messages across the package were disambiguated with call context.
- When a structural expression fails under the
data2mask (for examplesolve()of a singular matrix), the error now reports that cause instead of a misleading “not found” from the fallback lookup. - An unordered factor response to an ordinal family warns with the level order about to be used, then fits (the brms behavior);
mo()keeps its stricter ordered-factor requirement.
frmtmb 0.29.0
Review fixes for the spatial wave and rOpenSci review preparation.
Breaking
-
spde()grouping values are now explicitly MESH ROW INDICES: integer, whole-number character, and integer-level factor spellings are accepted and land identically; any other labels error stating the contract. Previously a non-mesh-order grouping (integer node IDs included, which sort lexicographically) fit a silently permuted field with a wrong likelihood and no warning. Code that passed non-numeric node labels must now map them to row indices.
Corrected behavior
-
car()/spde()accept call-valued grouping variables (gr = factor(node)). - The Windows sequential-chains guard in
frm_sample()also coversoptions(mc.cores =), which rstan reads as its default. - Optimizer error wrapping no longer claims a numerical cause for non-numerical errors; the underlying message leads.
- The
escarlog-determinant is finite atrhonear 1 (the adjacency eigenspectrum is clamped against floating-point overshoot). - The extreme-parameter heuristics in
diagnose()andfrm_sample()only test log-SD components and name the parameter; acar/bym2mixing proportion at its boundary no longer trips a misleading singularity warning. -
car()adjacency validation: NA entries and negative weights error informatively; one-sided dimnames work (the rownames-only path was broken); duplicate names error. - Non-finite covariance warnings fire once per fit instead of once per
vcov()call.
rOpenSci preparation
- Review infrastructure: CONTRIBUTING, code of conduct, CITATION, codemeta,
Languagefield, return-value documentation, README currency fixes. - srr statistical-software standards: General and Regression categories scaffolded with 77 standards tagged at their code locations, 11 marked inapplicable with justifications, and the 28 remaining gaps recorded as the submission roadmap in the standards file. Test coverage measured at 87.6% (full suite).
frmtmb 0.28.0
Spatial GMRF grammar, the mclust covariance taxonomy, and the close-out of the fuzz tier.
New
- Spatial conditional autoregressions with the brms spelling:
car(M, gr = g, type = "escar"/"esicar"/"icar"/"bym2")over an adjacency matrix, with analytic log-determinants throughout (no on-tape factorization) and brms’s per-component soft sum-to-zero constraint, whose distance to the hard-constrained ML is measured and documented (4e-4 logLik at the default, shrinking quadratically). Validated against hand-rolled direct ML to 1e-11. -
spde(fem, gr = node): an SPDE-Matern field from fmesher/INLA FEM matrices, Q(kappa, tau) assembled on the tape from fixed sparse matrices; sd and range reported through the planar identities. -
gr(g, prec = Q)supports correlated slopes (sparse Kronecker precisions; agrees with the densegr(cov = solve(Q))equivalent to 0 ulp). -
mixture_mvn(K, D, model =)gains the mclust covariance taxonomy (EII, VII, EEI, VEI, EVI, VVI, EEE, VVV) with per-model parameter templates; matchesmclust::Mclustto 1e-12 log-likelihood and 1e-15 posteriors on intercept-only fits, while keeping covariate-dependent means, which mclust cannot fit. -
anova(refit = TRUE)refits REML models with ML for comparison (warm-started, with a message naming what was refit);getME()on the lme4 generic with the commonly consumed vocabulary.
Corrected behavior
- Non-finite covariance matrices warn at
vcov()/summary()and at fit time underse = TRUE, pointing atdiagnose()(a positive-definite Hessian can still be numerically singular to invert; previously silent NaNs). - Optimizer failures recover automatically where a better start provably exists (the plain-Laplace optimum under
profile = TRUE; a cold start when a warm start is itself the failure), and every optimizer error carries the model label and remedies instead of a bare RTMB message. - One quadrature configuration is rescued by displaced calibrations; the six irrecoverable thin-data configurations refuse with messages naming the configuration and the guaranteed
quadrature = FALSEfallback. The fuzz tier runs green. -
frm_sample(cores > 1)on Windows falls back to sequential chains with a warning instead of failing incomprehensibly (RTMB tapes and objective closures cannot reach socket workers; upstream kaskr/tmbstan#27).pp_check()andas_draws()dispatch without attaching bayesplot or posterior (frmtmb ships its own generics, dual-registered).
Notes
- dev/benchmarks.md records the RTMBp (parallel RTMB) measurement: no gain on Cholesky-dominated mixed models, 1.7-2x on accumulation-dominated GLMMs at 4-16 threads; not adopted.
- dev/feature-gaps.md records the ODE roadmap (RTMBode) and the Suggests + Additional_repositories packaging plan for non-CRAN extensions.
frmtmb 0.27.0
Fixes from a full review of the v0.22-v0.25 waves.
Corrected behavior
-
cens()combined withtrunc()produced a silently wrong likelihood: the censoring contribution was not restricted to the truncation window (right-censoring now contributes (F(ub) - F(y))/Z), which inflated the dispersion by ~14% on the reference problem. The corrected objective matches a hand-rolled windowed likelihood to 1e-8, collapses exactly to the old form without truncation, and the cens+trunc OSA residuals recalibrate to the analytic PIT (5e-14). -
residuals(type = "deviance")underse()now weights each row’s unit deviance by its own variance (the glm prior-weight form) instead of treating a common dispersion as shared. -
predict(se.fit = TRUE)on quadrature fits no longer dies with a non-conformable error: it reports mode-conditional standard errors with a warning. Singular joint precisions in the predict path now degrade likevcov()does, and the shared solver usesMatrix::solve(strictly more robust for ill-conditioned but invertible GLMM precisions). -
frm_sample()chain inits are clamped inside any bounds passed to Stan, with a warning naming the parameters when the ML mode itself violates a bound (previously an incomprehensible rstan error). -
vint()/vreal()variables are required innewdata; a custom family’s prediction previously returned a length-0 vector with no message when they were missing. - Character censoring codes like “0”/“1”/“-1” decode (the error message had advertised them).
Compatibility registry and fuzzer
- The registry’s precedence is redesigned (lexicographic comparison of sorted side specificities, with a validator that forbids file-order ties except documented overrides). 488 of 3750 resolutions were corrected against probed reality, including 429 pairs whose covstruct conditions a family-level rule had erased, and the newly recorded caveat that spatial structures over a plain factor silently use level order as coordinates.
frm_compat()accepts vectors and errors on empty input. - The fuzz tier’s invariants were strengthened (a previously unfailable interval check replaced by the Wald identity plus parameter-coverage tallies; per-row simulate agreement; grammar divergences asserted; convergence demotion keyed to actual convergence verdicts after fixing a truncation that discarded them). The strengthened tier reproduces only known findings.
frmtmb 0.26.0
Pooled model comparison across imputations and the diagnostics/UX backlog.
New
-
anova()onfrm_multiple()fits pools nested-model tests across imputations: D1 (multivariate Wald), D2 (chi-square combining), and D3 (Meng-Rubin likelihood pooling, with the plug-in leg evaluated by re-taping each imputation’s objective at the pooled parameters - no refits). D1/D2 matchmiceto 1e-7 on an exactly-shared reference; D3 is validated against the Meng-Rubin formula directly, sincemice::D3’sfix.coefvariant is not the plug-in statistic. Includes an ARIV clamp and a Reiter-df fallback for a small-m case where mice returns NaN. -
cbind(successes, failures)binomial responses are accepted (the glm/lme4/glmmTMB spelling), rewritten internally tosuccesses | trials(successes + failures); bit-identical to thetrials()form. -
simulate()returns ordinal draws as ordered factors with the original levels and multinomial draws as count matrices (both families previously had no simulator), and respectsna.excludepadding. -
frmtmb_control(check_nlev_1 =, check_olre =): lme4-style warning/ignore/stop vocabulary for one-level grouping factors and gaussian observation-level random effects (these()-based meta-analysis idiom is recognized and not flagged). -
diagnose()adds a complete-separation heuristic, predictor-scale warnings pointing atautoscale, and an isSingular-style verdict independent of the Hessian; it also no longer errors on fits without random effects.
Corrected behavior
- Models with zero free outer parameters fit degenerately instead of dying inside nlminb (fixing latent empty-sdreport and empty-gradient bugs found along the way).
- A nonlinear-parameter name colliding with a data column errors instead of silently shadowing the column; nonlinear fits that fail from default zero starts name
start =in the error. - REML
anova()compares fits whose fixed-effect designs span the same column space (term reordering included) instead of refusing every REML pair; genuinely different designs and REML/ML mixes still refuse with the reason. - Offsets in distributional-parameter formulas were verified correct (to 1e-13) and are now regression-tested against the silent-drop failure mode reported upstream (glmmTMB#625).
frmtmb 0.25.0
Simulation-workflow ergonomics, the last deferred method-surface items, and CRAN readiness.
New
-
frm_simulate()accepts natural-scale parameter names - the same vocabulary ashypothesis()/variables(): coefficients by name,sd_<group>__<term>,cor_<group>__<t1>__<t2>, response-scalesigma- inverted to the internal parameterization through the covariance registry (us/diag/homdiag/smooth/gr structures; others refuse by block name). The internal spelling still works and is byte-identical. - Prior-predictive simulation, the
sample_prior = "only"analog:frm_simulate(..., priors = set_prior(...))draws a fresh parameter vector per replicate on each prior class’s documented scale (natural SDs for classsd, truncation by rejection) and attaches the drawn parameters for prior-predictive checks. Every coefficient and SD must be pinned by a prior ornewparams; omissions error instead of silently becoming zero effects. -
simulate(censored = TRUE)applies the fitted type-I censoring mechanism to the draws; the default remains the latent uncensored response, which is also brms’sposterior_predictconvention (verified against brms source) and is now documented. -
residuals(type = "deviance")across the GLM family set (gaussian, poisson, binomial/bernoulli, Gamma, exponential, inverse.gaussian, nbinom1/2, geometric, beta, tweedie), exact againststats::glmwhere glm offers them; families without a standard unit deviance refuse by name.deviance()itself stays-2 logLik(lme4 convention). -
predict(type = "response", se.fit = TRUE)now works for non-identity-mean families (zi/hurdle, lognormal, trials-binomial, truncated responses) through a joint delta method across all dpar linear predictors, including cross-dpar covariance (agrees with glmmTMB’s response-scale standard errors to 1.2e-5 on a zero-inflated poisson mixed model).
Corrected behavior
-
conditional_effects(method = "predict")evaluates addition terms on the effect grid, so its bands respecttrials()andtrunc()(binomial bands are counts in [0, n], not Bernoulli 0/1); aterm variables must be pinned inconditions, and the error names the variable.
CRAN and infrastructure
- Heavy reference-validation test files are
skip_on_cran()-gated: the CRAN-condition suite drops from ~128s to ~72s on the development machine while CI (NOT_CRAN=true) keeps full coverage. - nlme added to Suggests (a test uses
nlme::Soybeanas reference data; CI’s--as-cranunstated-dependency check halts without the declaration). - Benchmark verdict recorded in dev/benchmarks.md: optimParallel is not adopted - with exact AD gradients its concurrency caps at two evaluations (measured 1.03-1.22x end to end, slower with cold clusters), RTMB tapes cannot ship to PSOCK workers, and 100% of InstEval’s optimization time is inside the taped objective and the inner sparse Cholesky.
frmtmb_control(profile = TRUE)remains the measured lever for many-coefficient models (1.6x there).
frmtmb 0.24.0
The quadrature and OSA defect clusters surfaced by the fuzzer and compatibility registry.
Corrected behavior
-
quadrature = TRUEis rebuilt around one root cause: TMB’s Gauss-Kronrod marginalization calibrates each integrand’s rescaling once, at the parameter values in hand when the tape is built, and frmtmb taped at the cold start. A plain Laplace fit now runs first and the quadrature tape is built at its optimum. This fixes: conditional modes returned as NA (ranef()/fitted()/predict()were silently NA for all groups but the first, whose slot held a wrong value) - modes now come from the Laplace inner solve and matchglmer(nAGQ = 25)’sranef()to 3e-5; and bare “NA/NaN gradient evaluation” crashes for poisson/Gamma/Beta with nested or even single scalar blocks - all now fit with gradients < 3e-4. -
quadraturecombined withtrunc()produced logLik = +Inf as a successful fit; the combination is refused (the CDF difference underflows at quadrature nodes; a stable fix needs log-CDF forms).mixture()withREMLorprofile = TRUEis refused: both Laplace-expand the mu coefficients around a single mode, and a mixture likelihood is permutation-multimodal in exactly those coefficients.mixture()with quadrature remains supported. -
residuals(type = "osa")on censored fits was broken (raw LAPACK singularity or NaN for every censored row). Uncensored rows now get calibrated one-step residuals by conditioning on the censoring window and renormalizing the one-step CDF to it (matches the analytic conditional PIT to 7e-9); censored rows are NA (an event has no one-step CDF), and row-varying censoring points refuse. Notesimulate()draws the latent uncensored response undercens(), so DHARMa is not a substitute there (documented). -
residuals(type = "osa")on ordinal fits crashed inside the OSA machinery; the ordinal log-densities now handleoneStepPredict’s taped-observation objects (exact Lagrange-basis category indicator), and all four ordinal families produce calibrated residuals (cumulative matches the analytic randomized-quantile residual to 4e-14). - A singular joint precision under REML or
profile = TRUEno longer throws a raw LAPACK error fromvcov()/summary()/confint()/hypothesis(): it degrades to NaN standard errors with one warning pointing atdiagnose(), matching the ML branch. - The brms-migration vignette documents that
binomial()withouttrials()means Bernoulli here (glm convention) where brms requirestrials()orbernoulli().
The fuzz tier’s open findings drop from 28 to 16 on the identical plan; most of the remainder are now informative refusals on thin-data edge cases rather than defects.
frmtmb 0.23.0
Defect wave driven by an open-issue sweep of brms/lme4/glmmTMB, plus a feature-compatibility registry.
Corrected behavior
- Truncation now reaches the whole post-fit surface, not just the likelihood:
fitted(),predict(type = "response"), andresiduals()report the truncated mean E[Y | lb <= Y <= ub] (closed forms for all six CDF families, validated to 1e-15);simulate(),posterior_predict(), andfrm_simulate()rejection-sample within bounds; newdata re-evaluates variable bounds.residuals(type = "osa")on truncated (and untruncated gaussian) responses was also miscalibrated and now integrates over the truncated support (KS uniformity restored from p ~ 1e-15 to 0.76-0.97). Dpar-scale predictions stay untruncated by design. DHARMa/pp_check on truncated models are meaningful again. -
(f || g)with a factor now yields independent per-level effects (the diag structure the syntax promises) instead of a silently fully correlated block; identical to an explicitdiag(f | g). Numeric double-bars are unchanged. Existing factor-double-bar fits change, because the old ones were wrong (lme4 has the same open bug, lme4#818). -
ar1()/hetar1()warn when the ordering factor’s integer levels have gaps: levels correlate by position (the glmmTMB reading, unchanged), so a gap counts as one step; the warning points atou()overnum_factor()for irregular spacing (glmmTMB#1278). - Predictions at non-estimable points of a rank-deficient design return NA with one warning naming the aliased columns, instead of silently returning the partial sum (predict.lm semantics; lme4#303). Collinear restatements of kept columns stay exact.
- Grouping factors written as calls work:
(1 | factor(x)),(1 | interaction(a, b))(lme4#464). - A random-effect term crossed with
*or:errors instead of being silently refit additively (lme4#196);mo()/mi()interaction multipliers must be numeric (brms#1828);anova()requires equalnobs(lme4#622). -
frm_sample()chain initialization: chain 1 anchors at the ML mode, further chains are jittered on the unconstrained scale (init_jitter), restoring the overdispersion Rhat needs; a boundary-mode warning fires for singular fits; mixture posteriors are documented as needinginit = "random".
New
- Feature compatibility registry:
frm_compat()answers what plays with what (works / conditional / refused / broken / untested) across 3750 feature pairs, and the new “Feature compatibility” article is generated from the registry at build time so it cannot drift. Known-broken pairs are listed there; the registry probing itself surfaced trunc x quadrature and cens x OSA as broken (queued for the next fix wave). - An audit of 559 currently open brms/lme4/glmmTMB issues is recorded in dev/test-backlog.md, including the pathologies frmtmb is structurally immune to.
- A pairwise grammar fuzzer (env-gated:
FRMTMB_FUZZ=true): a 310-spec covering array over the grammar with metamorphic invariants (predict/fitted identity, permutation invariance, simulator support membership, vcov sanity) and a brmsmake_standatastructural oracle. Its first run surfaced the quadrature defect cluster now queued for fixing (conditional modes left NA, crashes with nested groups and Beta, +Inf logLik with trunc()); the tier reports those as failures until they are fixed.
frmtmb 0.22.0
Cross-validation against brms itself, closer brms compatibility, and fit ergonomics.
brms agreement suite
- New test tier validating the elaborate grammar structurally against
brms::make_standata()/brmsterms()without any Stan compilation (designs, RE structures,|ID|merging, ordinal thresholds,cs(),mo()codes, gp bases, smooths, addition terms, mvbf, nl, mixture naming; most exact to 1e-10 or better), plus an opt-in numeric tier (FRMTMB_BRMS_FIT_TESTS=true) showing fixed-effects estimates match the mode of brms’s own generated Stan program to 1e-4 and that ourmo()parameterization is brms’s likelihood exactly (viarstan::log_prob). brms is in Suggests only. - Found by the suite and fixed:
cs()predictor variables never reached the combined model frame, sobf(y ~ x + cs(z))errored unlesszalready appeared elsewhere.
brms compatibility
- Hilbert-space
gp(x, k =)now uses brms’s input convention exactly: coordinates are rescaled by the largest pairwise distance (one shared factor across dimensions) and centered, so the boundary isL = cand the samegp()call is the same approximation in both packages (basis agreement with brms to 1e-16, tied coordinates and vector-valuedc =included). This roughly doubles the effective boundary at a givenc: accuracy against the exact gp improves about 25x at the same k (k = 40 now within 2e-4 logLik), and the defaultc = 1.25is the right choice in 2-D as well. Reportedrange(gp)values stay in data units (exact log-shift back-transform); fitted lengthscales differ from pre-0.22 fits by the data-dependent scale factor. -
cens()accepts brms’s character and factor codes ("left"/"none"/"right"/"interval", prefix-matched, case-insensitively); unknown labels and out-of-range numeric codes error informatively instead of coercing to NA. -
gp()’sk/c/iso,rr()’sd, andse()’ssigmaarguments now evaluate in the formula environment (brms behavior), so variables and expressions work; invalid values error with the offending expression named.
Fit ergonomics
-
frmtmb_control(verbose =)(and thefrm(verbose =)shortcut): level 1 prints timed stage lines (parse, frame, tape, optimize, restarts, sdreport) throughmessage()so a slow fit shows where it is slow; level 2 adds the optimizer’s own iteration trace. Bootstrap, influence, and allfit refit loops stay quiet. - The large-gradient warning now points to
diagnose()and the new “Convergence problems” remedy ladder in the diagnostics vignette (scaling, restarts, optimizer comparison, profiles, boundary fits, and judging marginal gradients). -
frm_allfit()’s nloptr optimizer produced an empty row: nloptr rejects callbacks that declare...(RTMB’sobj$fn/obj$grdo), and the error was swallowed; separately, missingftol_relmade NLopt report failure at the converged optimum. Both fixed; the agreement test now requires every optimizer to converge and match. - The brms-migration vignette documents how to recover the model function (the
stancode()analog): the objective closure viabuild_objective(fit$frame), its joint-vs-marginal relationship tofit$obj$fn, and how to reproducefit$obj.
frmtmb 0.21.0
Method-surface audit against lme4/glmmTMB/brms, plus fixes from a full code review.
Corrected behavior
-
predict(type = "response")now returns the expected response for every family (equal tofitted()in-sample), not the primary dpar’s natural scale; zi/hurdle, lognormal, and trials-binomial fits were affected. The glmmTMB aliases"conditional","zprob","zlink", and"disp"are accepted (validated against glmmTMB on a zero-inflated Poisson to ~8e-7). Note"disp"returnssigmaon its natural scale, where glmmTMB returns the variance-scale dispersion for gaussian.se.fitfor the mean of a non-identity-mean family is not yet available and errors with guidance. -
rescor = TRUEnow refusescens(),trunc(), andse()addition terms, which the joint-gaussian likelihood silently ignored (wrong likelihood with no warning). - Bounds (
lower/upper,set_priorlb/ub) underfrmtmb_control(profile = TRUE)were positionally misaligned: a bound on a fixed coefficient could silently pin a covariance parameter instead. Bounding a profiled coefficient now errors; covariance-parameter bounds land on the right slot. -
confint(),vcov(full = TRUE), anddiagnose()labels were broken or misaligned onmi()fits: the latent imputation component was counted as an outer parameter. -
frm_sample(priors = )failed on fixed-effects-only fits of single-dpar families (a$partial-match bug), andfrm_sample(laplace = TRUE)was broken twice: the default init had the wrong length, and draw columns were mislabeled (a column namedb[1]actually held a covariance parameter). -
influence()tables andcooks.distance()now align withvcov()(fits with a constant dpar errored; column names are now the coefficient labels). -
simulate()follows the stats seed contract: a"seed"attribute and a restored RNG state instead of clobbering the global stream. - A covariate literally named
sigmais no longer shadowed by the residual sigma inhypothesis()/variables(). - In
predict(newdata = ), an NA in a variable used only in a random-effect design now propagates to the prediction instead of being silently zeroed (only genuinely new levels predict at the population value).
New methods
-
drop1()(AIC/LRT, marginality-aware; matcheslme4::drop1.merModto 1e-4),cooks.distance()directly on a fit,dfbeta()/dfbetas()oninfluence()results (stats sign convention; lme4 returns the negation),na.action(), and an lme4-style “Groups:” line insummary(). -
confint()accepts lme4’s"Wald"spelling and gainsmethod = "boot"(percentile intervals via [frm_bootstrap()]). -
vcov(full = TRUE)rows carry per-parameter names (glmmTMB convention). -
predict()accepts the lme4/glmmTMBallow.new.levelsdot spelling. - On draws objects:
posterior_linpred()andranef()(brms-shaped arrays). -
emmeans::recover_datahad been silently missing from NAMESPACE since v0.13 (orphaned export tag); emmeans support was broken.
Internals
- One authoritative outer-parameter map shared by
confint(), bounds resolution,vcov(full = TRUE), and sampling;graphicsandgrDevicesadded to Imports; duplicated formula/family coercion and gp position-key helpers unified; REMLlogLikdf counts only outer parameters (documented; lme4 counts the integrated fixed effects too, so REML AICs are not comparable across packages).
frmtmb 0.20.0
-
mixture_mvn(K, D): multivariate gaussian mixture components for model-based clustering of an n x D matrix response (the mclust / clustTMB use case). Every class mean is a full linear predictor, so cluster means may depend on covariates and random effects; mixing weights are multinomial-logit dpars with their own formulas. Validated against a hand-rolled ML fit to 1.3e-10 and against faithful-data cluster recovery. Limitations (documented in?mixture_mvn): class covariances are unstructured (us) and covariate-free - no mclust-style constrained covariance taxonomy (EII..VEV);cens()/trunc(),mvbf(), andsimulate()are not supported. - Multi-dimensional Gaussian processes:
gp(x1, x2, ...)takes up to three variables, with a separate lengthscale per dimension by default (the brms convention) or one shared lengthscale viaiso = TRUE, in both the exact and the Hilbert-space (k =) form. The tensor HSGP basis is capped atk^D <= 1000columns.confint_varcorr()reports one range row per dimension. Validated against a direct 2-D ML fit to 3.8e-11. - Kriging prediction for the exact
gp():predict(newdata = )at positions not seen at fit time now returns the GP conditional mean, andse.fit = TRUEadds the conditional variance, instead of erroring. Kriging mean validated to 1.7e-15 and the standard error to the exact decomposition identity. The Hilbert-space form already interpolated through its basis. -
frmtmb_control(sparse_x = TRUE): sparse fixed-effect design matrices (Matrix::sparse.model.matrix), the analog ofglmmTMB(sparseX = ), for models where a many-level fixed factor dominates memory. Estimates are identical to the dense path (gradient agreement 3e-14; 13.8% lower tape memory in the validation model);model.matrix()on the fit returns a sparse matrix. Prediction falls back to the dense builder for newdata containing NA factor rows, whichsparse.model.matrixwould silently zero. -
frmtmb_control(autoscale = TRUE): internal standardization of badly scaled continuous predictors (the lme4 >= 1.1.37 feature): a scaled pre-fit is mapped back exactly and warm-starts the reported fit, parameter magnitudes feednlminb’s scale hook, and convergence and standard-error Hessian checks run in scaled units. Turns 12 scaling warnings into 0 on the validation problem with a 0.0 log-likelihood difference against a manually standardized reference. Reported results are always on the original scale. -
frm_multiple()pooling extensions:$pooled_varcorrpools random-effect SDs and correlations across imputations on transformed scales (log for SDs and GP ranges, Fisher z for correlations) with Barnard-Rubin degrees of freedom, andhypothesis()on afrmtmb_multipleobject pools delta-method hypothesis estimates by Rubin’s rules with t-based intervals. Fixed-effect pooling matchesmice::pool()to 2.1e-6. Likelihood-ratio pooling across imputations (mice’s D3) is not implemented;anova()on pooled fits remains per-fit.
frmtmb 0.19.0
- Latent classes now combine with continuous random effects (growth-mixture models):
mixture(..., groups = ~g)accepts random effects, smooths, and gp() terms in the component formulas. The class sum is computed conditional on the latent effects, so one Laplace approximation integrates them - the sum-and-integral swap makes this exact as an integrand identity, and even crossed random effects are structurally fine. Random effects written in a component formula are class-specific by construction. Accuracy: the Laplace approximation of the class-mixture integrand carries a small documented bias (about 0.1 log-likelihood units in the validation problem, with parameter estimates matching the exact closed-form marginal to 0.01);quadrature = TRUEis numerically exact when the per-group integrand is univariate (validated to 5e-3) and approximate when class-specific intercepts couple.simulate()now draws one class per group and then simulates each observation from its group’s component;mixture_probs()gives empirical-Bayes classification conditional on the modes. -
variables()(brms spelling): lists every parameter name usable inhypothesis()expressions (coefficients,sd_/cor_summaries,sigma); onfrm_sample()output it lists the draw columns. -
get_prior()(brms spelling): enumerates every slotset_prior()can target (class/coef/dpar/group rows), from a formula plus data or from a fit; the default in every slot is flat. - README rewritten for the current scope, with related-work pointers (glmmTMB, BayesRTMB); getting-started vignette extended to the full grammar; SPEC.md status and deviations brought current.
frmtmb 0.18.0
The last three roadmap features.
-
gp(x)Gaussian-process terms: exact (a dense squared-exponential block over the unique positions, with a standard 1e-6 nugget) or the Hilbert-space approximation viagp(x, k = 30)(sine basis with spectral-density prior SDs), in any linear predictor. Exact fits match direct GP marginal ML; the approximation converges to the exact answer and predicts at arbitrary positions (the exact form predicts at observed positions only).confint_varcorr()reportssd(gp)andrange(gp). -
mo()andmi()interactions:mo(x):z,mo(x)*z,mi(x):z,mi(x)*zwith numeric multipliers;mo()interactions share their variable’s simplex (brms convention). Both validated exact against direct ML (themi()interaction stays linear in the latent value, so the closed-form marginal still applies). - Group-level latent-class mixtures:
mixture(..., groups = ~g)sums the class assignment per group (growth-mixture / latent-class regression; the tractable nested case of brms#1905). Exact against direct ML;mixture_probs()returns posterior class probabilities per group, or per observation for ordinary mixtures. Restrictions: no random effects or smooths alongside, mixing predictors evaluated at each group’s first row, nosimulate()yet. Crossed-design group mixtures remain out of the Laplace class.
frmtmb 0.17.0
Clearing the roadmap: mixtures, measurement error, and the remaining grammar gaps.
-
mixture(fam1, fam2, ...)finite-mixture families: each component keeps its own suffixed dpars (mu1,sigma1, …), mixing proportions are multinomial-logit dpars with their own linear predictors (mixture-of-experts comes free), and the likelihood is a branch-free logsumexp - so random effects and smooths work in any component formula. Exact against direct ML for gaussian and poisson mixtures. Component means initialize on spread response quantiles; the usual finite-mixture multimodality caveats apply (frm_allfit(), or order intercepts via bounds). -
mi(sdx)measurement error (brmsme()): known per-observation measurement SDs make every true value latent, with the observed values entering through a measurement model. Exact against the closed-form bivariate-normal marginal; attenuation bias is corrected and NAs (missing + mismeasured) combine. -
cs()category-specific effects forsratio,cratio, andacat(refused forcumulative, as in brms). Exact against direct ML. -
equalto(x + 0 | g, V)covariance structure: fully fixed known V, zero parameters (meta-analytic sampling covariances). - rr() fits now support
se.fit: the delta method routes the factor columns through the loadings Jacobian and adds loading-parameter columns; verified parameterization-invariant against the full-rank-vs-us()equivalence. - Registered insight methods (
find_formula,find_random,get_parameters,get_varcov,find_statistic, link accessors):insight::is_mixed_model()and the easystats accessor layer now see the random-effect structure.
frmtmb 0.16.0
The two deferred architecture features.
-
rr(x | g, d = k)reduced-rank / factor-analytic covariance (glmmTMB’s GLVM structure): per-level coefficients are loadings times iid standard-normal factors, sobholds the factors while the design matrices span the coefficient space. Matches glmmTMB exactly and reduces tous()at full rank.se.fiton rr models is deferred (the loadings enter the coefficients nonlinearly);predict,simulate,ranef, andVarCorr(the rank-k covariance) all work. -
mi()one-step imputation for continuous predictors, reversing the original exclusion:bf(y ~ mi(x) + z) + bf(x | mi() ~ z)turns missingxvalues into latent parameters that Laplace integrates (exactly, in the linear-gaussian case - validated against the closed-form marginal likelihood). Imputation uncertainty propagates into the coefficient standard errors automatically. Rows with NAs in non-mi()variables still drop; imputation models must be gaussian or student; discrete predictors remain impossible (as in Stan).me()measurement error is the natural follow-on.
frmtmb 0.15.0
Tier-2 sweep of dev/feature-gaps.md plus a method surface for sampled fits.
-
mo()monotonic effects (brms syntax): scale coefficient times an estimated simplex, exact against direct ML; works in prediction,conditional_effects(), and with ordered factors. No frequentist package offers this. Standalone additive terms only for now. - Sequential ordinal families
sratio,cratio,acat, validated against direct ML and collapsing to logistic regression at K = 2. - Covariance structures:
hetar1,homcs,homtoep(glmmTMB parameterizations, exact where glmmTMB converges) and spatialexp(),gau(),mat()overnum_factor(x, y)coordinates (Matern uses bounded internal transforms for stability where both we and glmmTMB otherwise diverge). - Fixed a latent parser bug:
exp(x)(or any covariance-structure name) used as a plain function in a fixed formula was silently stripped by the formula splitter - since v0.1. Such calls are now protected automatically. -
influence(fit, groups = )andcooks.distance(): case-deletion diagnostics over warm-started refits. -
frm_multiple(): fits across multiply-imputed datasets (list ormice::mids) pooled by Rubin’s rules with Barnard-Rubin df. -
conditional_effects()gainsmethod = "predict"(prediction intervals) and data-frameconditions(one condition set per row, brms style). -
vint()/vreal()addition terms pass arbitrary data vectors to custom families (brms custom-family convention). The test suite includes a full Wiener drift-diffusion model written as acustom_family()in plain R - the workflow brms needs raw Stan code for. - Method surface for
frm_sample()draws:summary()(with Rhat/ESS),fixef(),VarCorr()(natural scale),hypothesis()(exact posterior version),posterior_epred()/posterior_predict()(each draw runs the full prediction machinery),pp_check(), andposterior::as_draws(). - Examples added across the reference (run by R CMD check).
frmtmb 0.14.0
Gap-closing milestone from a sweep of the lme4 / glmmTMB / brms vignettes (dev/feature-gaps.md).
-
se()addition term: known per-observation sampling SDs (meta-analysis), gaussian and student.se(x)fixes the residual SD (sigma is mapped out);se(x, sigma = TRUE)adds an estimated sigma in quadrature. With(1 | obs)this is random-effects meta-analysis, and withgr(g, cov = A)the phylogenetic version. - Binomial (and beta-binomial) responses may be proportions of
trials(), the glm/glmer idiom, converted to counts internally. - New families:
bernoulli,geometric,exponential,weibull(mean-parameterized, matches survreg’s likelihood),shifted_lognormal,hurdle_gamma,hurdle_lognormal,zero_inflated_binomial,zero_inflated_beta(validated against glmmTMB), andasym_laplacefor quantile regression (fixedquantiledpar reproducesquantreg::rqestimates). -
ranef(condVar = TRUE): conditional SDs of the modes (TMB/glmmTMB convention: fixed-effect uncertainty propagates), plus tidyas.data.frame()methods forranefandVarCorroutput. -
frm_allfit(): refit under every available optimizer (nlminb, optim, bobyqa, NLopt) and compare, the lme4allFit()analog. -
frm_simulate(): de novo simulation from abf()formula and supplied parameters with no fitted model (power analysis; the glmmTMBsimulate_new()analog). -
frmtmb_control(profile = TRUE)(experimental): profile the primary-coefficient vector into the inner Laplace problem, theglmmTMBControl(profile = TRUE)/glmer(nAGQ = 0)speed approximation for many-coefficient models; covariance comes from the joint precision. -
anova()no longer fails on models sharing a primary formula (distributional vs plain fits); row labels now include dpar formulas.
frmtmb 0.13.0
Method-surface (“sugar”) milestone: the conventional S3 methods that downstream packages and user muscle memory dispatch on.
- New accessors:
sigma(),terms(),weights(),model.matrix(),deviance(),extractAIC()(enablesstep()/drop1()),ngrps(),prior_summary(), and aprofile()method wrappingTMB::tmbprofile(). -
Breaking:
coef()now follows the lme4 / glmmTMB / brms convention (per-group coefficients: fixed effects broadcast over levels plus the conditional modes). Usefixef()for the fixed effects alone. Fits without random effects still return the coefficient vector. -
conditional_effects()with a base-graphicsplot()method: grid predictions per predictor (or"x:z"pair) with Wald bands, smooths included, matrix covariates held at column means. -
plot(fit): Pearson-residual diagnostics (residuals vs fitted, normal QQ). -
pp_check()(registered on the bayesplot generic): predictive checks fromsimulate()draws through anyppc_*function. -
hypothesis(): tests of arbitrary expressions of the parameters, brms spelling, with three methods: delta-method Wald (default),"profile"(profile-likelihood intervals viaTMB::tmbroot()lincombs; linear hypotheses, ML fits), and"boot"(parametric bootstrap percentile intervals; any expression). The expression environment includes natural-scale random-effect names (sd_<group>__<term>,cor_<group>__<t1>__<t2>,sigma), so ICC inference ishypothesis(fit, "sd_g__Intercept^2 / (sd_g__Intercept^2 + sigma^2)", method = "boot"). The result is afrmtmb_hypothesisdata frame carrying the bootstrap draws or profile curves in attributes, with aplot()method (bootstrap histogram, profile curve, or the implied Wald normal, per hypothesis). -
frm_bootstrap(fit, FUN, nsim): parametric bootstrap overrefit()s (thebootMeranalog), withconfint()percentile intervals. -
refit(fit, newresp): refit to a new response reusing the assembled design with a warm start - one re-tape plus optimization; the engine for parametric bootstrap oversimulate()draws. - The accessor set makes insight’s default methods work (
find_response(),get_data(),get_sigma(),n_obs(), …); dedicated insight methods (random-effect formula parts,get_parameters()) are a roadmap item.
frmtmb 0.12.0
Pre-release consolidation toward CRAN. Highlights by milestone:
Model grammar (brms-compatible spelling)
-
bf()with distributional-parameter formulas carrying the full predictor grammar (random effects and smooths in any dpar), constant dpars, and addition termsweights(),trials(),cens()(left / right / interval),trunc()(with the inclusive discrete correction). - Nonlinear formulas (
nl = TRUE) with a full linear predictor per parameter. - Multivariate models:
mvbf()/bf() + bf()/mvbind(), a family per response, gaussian residual correlation (set_rescor()), and|ID|random-effect correlation across formulas. - mgcv smooths
s()/t2()in any linear predictor, including matrix-covariate summation-convention terms: scalar-on-function, function-on-scalar, and function-on-function regression validated against mgcv. - Covariance structures:
us,diag,homdiag,cs,ar1,toep,ou(positions vianum_factor()), and known-structure termsgr(g, cov = A)(dense, with correlated slopes via a Kronecker product) andgr(g, prec = Q)(sparse GMRF).
Families
gaussian, poisson, binomial, Gamma, lognormal, student, negbinomial, nbinom1, beta, tweedie, compois, beta_binomial, skew_normal, inverse.gaussian, exgaussian, zero-inflated and hurdle counts, cumulative ordinal, matrix-response multinomial, and custom_family() as a plain R log-density with check_custom_family() AD verification.
Estimation and inference
- ML and REML by Laplace approximation;
quadrature = TRUEupgrades scalar random effects to adaptive Gauss-Kronrod marginalization (matchesglmer(nAGQ = 25)and GLMMadaptive). - MAP / regularized ML via brms-style
set_prior(); hard bounds (lower/upper, and per-priorlb/ub). -
sdreportis deferred until standard errors are first needed (roughly a quarter off fit time;se = TRUErestores eager mode). - Pluggable optimizers (
frmtmb_control(optimizer = )), including arbitrary user functions. -
confint()(Wald / profile / likelihood-root), natural-scaleconfint_varcorr(),anova()likelihood-ratio tests,diagnose().
Diagnostics and ecosystem
- Residuals: response, pearson, and one-step-ahead (
type = "osa"). - DHARMa simulation-based residuals (
dharma_residuals()). - NUTS on the fitted objective:
frm_sample()(ML-mode initialization, named draws, priors, bounds) andcheck_laplace(); bayesplot works on the draws. - emmeans and marginaleffects support, both verified against glmmTMB.