4
$\begingroup$

I’m new to statistical modelling, and I’ve managed to pick quite a complicated first problem.

Essentially, I have a bivariate linear mixed effects model, where I am trying to model an imaging parameter I (over patient age) against a neurological outcome N (over patient time since diagnosis), where there are shared covariates between I and N. I also have random effects for the patients, specifically for y-intercept and slope. The model converges, everything looks good, but what I am now trying to do is estimate how well the model will perform on future data.

I’ve been using cluster bootstrapping in R to calculate the optimism-corrected R2 values, as well as calibration curves. I think my main question is: Should I only do optimism-correction for marginal (fixed effects) R2 and calibration slope? Or, should I include optimism-correction for conditional (fixed+random effects) R2 and calibration slope? I’m asking because I’ve found that it is relatively straightforward to set up optimism-correction for marginal R2 and calibration slope, but I’ve not been able to sort it for conditional R2 and calibration slope.

Thank you, and apologies for my ignorance on this!

E

New contributor
Ad Animum is a new contributor to this site. Take care in asking for clarification, commenting, and answering. Check out our Code of Conduct.
$\endgroup$

1 Answer 1

3
$\begingroup$

Yes that is an extremely complex problem to face when being introduced to statistical models. I use the Efron-Gong bootstrap optimism correction all the time to get overfitting-corrected model performance measures including smooth nonparametric calibration curves, as described here. I have not applied that method to random effects, but model calibration refers to the whole model and not just pieces of it, so I think that the cluster bootstrap in conjunction with estimated linear predictors that include random effects should work OK.

$\endgroup$

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.