UC Irvine

Prediction of clinical outcomes is of primary scientific interest in many public health studies. While prediction assessment tools have been developed and refined for simple random samples and characterized by non-missing outcome data, modern epidemiologic datasets bring unique challenges to the assessment and selection of prediction models. These challenges include high-dimensional predictor spaces, non-random sampling, and censored and longitudinal outcomes. In this dissertation, we consider prediction assessment that incorporates these complexities. We first consider prediction in a high-dimensional predictor space setting with non-random sampling of study participants. We empirically illustrate the limitations of the weighted ridge regression estimator in this case and propose a novel estimator that allows for the adjustment of sampling weights in the ridge regression penalty structure to provide more generalizable predictions. We then consider an analytic estimate of the out-of-sample prediction error for regression based censored survival models, where performance is measured via the Brier score. We derive an analytic estimate of out-of-sample error under single population models with uncensored data and extend this to propose an algorithm for prediction assessment under Cox regression with covariate adjustment. Finally we consider the marginal prediction of a longitudinal process impacted by both differential follow-up and non-random sampling. We derive a cluster-wise re-sampling framework that incorporates unbalanced sampling at the subject level and within-subject correlation. We empirically illustrate the utility of the proposed framework via simulation. Throughout, the methods are applied to data predicting cognition in the setting of Alzheimer’s disease and to the prediction of access graft failure among hemodialysis patients.