Objectives

The aim of this study was to evaluate whether machine learning (ML) of noncontrast computed tomographic (CT) and clinical variables improves the prediction of atherosclerotic cardiovascular disease (ASCVD) and coronary heart disease (CHD) deaths compared with coronary artery calcium (CAC) Agatston scoring and clinical data.

Background

The CAC score provides a measure of the global burden of coronary atherosclerosis, and its long-term prognostic utility has been consistently shown to have incremental value over clinical risk assessment. However, current approaches fail to integrate all available CT and clinical variables for comprehensive risk assessment.

Methods

The study included data from 66,636 asymptomatic subjects (mean age 54 ± 11 years, 67% men) without established ASCVD undergoing CAC scanning and followed for cardiovascular disease (CVD) and CHD deaths at 10 years. Clinical risk assessment incorporated the ASCVD risk score. For ML, an ensemble boosting approach was used to fit a predictive classifier for outcomes, followed by automated feature selection using information gain ratio. The model-building process incorporated all available clinical and CT data, including the CAC score; the number, volume, and density of CAC plaques; and extracoronary scores; comprising a total of 77 variables. The overall proposed model (ML all) was evaluated using a 10-fold cross-validation framework on the population data and area under the curve (AUC) as metrics. The prediction performance was also compared with 2 traditional scores (ASCVD risk and CAC score) and 2 additional models that were trained using all the clinical data (ML clinical) and CT variables (ML CT).

Results

The AUC by ML all (0.845) for predicting CVD death was superior compared with those obtained by ASCVD risk alone (0.821), CAC score alone (0.781), and ML CT alone (0.804) (p < 0.001 for all). Similarly, for predicting CHD death, AUC by ML all (0.860) was superior to the other analyses (0.835 for ASCVD risk, 0.816 for CAC, and 0.827 for ML CT; p < 0.001).

Conclusions

The comprehensive ML model was superior to ASCVD risk, CAC score, and an ML model fitted using CT variables alone in the prediction of both CVD and CHD death.

Background

Machine learning (ML) models can improve prediction of major adverse cardiovascular events (MACE), but in clinical practice some values may be missing. We evaluated the influence of missing values in ML models for patient-specific prediction of MACE risk.

Methods

We included 20,179 patients from the multicenter REFINE SPECT registry with MACE follow-up data. We evaluated seven methods for handling missing values: 1) removal of variables with missing values (ML-Remove), 2) imputation with median and unique category for continuous and categorical variables, respectively (ML-Traditional), 3) unique category for missing variables (ML-Unique), 4) cluster-based imputation (ML-Cluster), 5) regression-based imputation (ML-Regression), 6) missRanger imputation (ML-MR), and 7) multiple imputation (ML-MICE). We trained ML models with full data and simulated missing values in testing patients. Prediction performance was evaluated using area under the receiver-operating characteristic curve (AUC) and compared with a model without missing values (ML-All), expert visual diagnosis and total perfusion deficit (TPD).

Results

During mean follow-up of 4.7 ± 1.5 years, 3,541 patients experienced at least one MACE (3.7% annualized risk). ML-All (reference model-no missing values) had AUC 0.799 for MACE risk prediction. All seven models with missing values had lower AUC (ML-Remove: 0.778, ML-MICE: 0.774, ML-Cluster: 0.771, ML-Traditional: 0.771, ML-Regression: 0.770, ML-MR: 0.766, and ML-Unique: 0.766; p < 0.01 for ML-Remove vs remaining methods). Stress TPD (AUC 0.698) and visual diagnosis (0.681) had the lowest AUCs.

Conclusion

Missing values reduce the accuracy of ML models when predicting MACE risk. Removing variables with missing values and retraining the model may yield superior patient-level prediction performance.