Sepsis is a leading cause of morbidity and mortality worldwide. Early identification of sepsis is important as it allows timely administration of potentially life-saving resuscitation and antimicrobial therapy. We present COMPOSER (COnformal Multidimensional Prediction Of SEpsis Risk), a deep learning model for the early prediction of sepsis, specifically designed to reduce false alarms by detecting unfamiliar patients/situations arising from erroneous data, missingness, distributional shift and data drifts. COMPOSER flags these unfamiliar cases as indeterminate rather than making spurious predictions. Six patient cohorts (515,720 patients) curated from two healthcare systems in the United States across intensive care units (ICU) and emergency departments (ED) were used to train and externally and temporally validate this model. In a sequential prediction setting, COMPOSER achieved a consistently high area under the curve (AUC) (ICU: 0.925-0.953; ED: 0.938-0.945). Out of over 6 million prediction windows roughly 20% and 8% were identified as indeterminate amongst non-septic and septic patients, respectively. COMPOSER provided early warning within a clinically actionable timeframe (ICU: 12.2 [3.2 22.8] and ED: 2.1 [0.8 4.5] hours prior to first antibiotics order) across all six cohorts, thus allowing for identification and prioritization of patients at high risk for sepsis.

Background

Prediction-based strategies for physiologic deterioration offer the potential for earlier clinical interventions that improve patient outcomes. Current strategies are limited because they operate on inconsistent definitions of deterioration, attempt to dichotomize a dynamic and progressive phenomenon, and offer poor performance.

Objective

Can a deep learning deterioration prediction model (Deep Learning Enhanced Triage and Emergency Response for Inpatient Optimization [DETERIO]) based on a consensus definition of deterioration (the Adult Inpatient Decompensation Event [AIDE] criteria) and that approaches deterioration as a state "value-estimation" problem outperform a commercially available deterioration score?

Derivation cohort

The derivation cohort contained retrospective patient data collected from both inpatient services (inpatient) and emergency departments (EDs) of two hospitals within the University of California San Diego Health System. There were 330,729 total patients; 71,735 were inpatient and 258,994 were ED. Of these data, 20% were randomly sampled as a retrospective "testing set."

Validation cohort

The validation cohort contained temporal patient data. There were 65,898 total patients; 13,750 were inpatient and 52,148 were ED.

Prediction model

DETERIO was developed and validated on these data, using the AIDE criteria to generate a composite score. DETERIO's architecture builds upon previous work. DETERIO's prediction performance up to 12 hours before T0 was compared against Epic Deterioration Index (EDI).

Results

In the retrospective testing set, DETERIO's area under the receiver operating characteristic curve (AUC) was 0.797 and 0.874 for inpatient and ED subsets, respectively. In the temporal validation cohort, the corresponding AUC were 0.775 and 0.856, respectively. DETERIO outperformed EDI in the inpatient validation cohort (AUC, 0.775 vs. 0.721; p < 0.01) while maintaining superior sensitivity and a comparable rate of false alarms (sensitivity, 45.50% vs. 30.00%; positive predictive value, 20.50% vs. 16.11%).

Conclusions

DETERIO demonstrates promise in the viability of a state value-estimation approach for predicting adult physiologic deterioration. It may outperform EDI while offering additional clinical utility in triage and clinician interaction with prediction confidence and explanations. Additional studies are needed to assess generalizability and real-world clinical impact.

Sepsis remains a major cause of mortality and morbidity worldwide. Algorithms that assist with the early recognition of sepsis may improve outcomes, but relatively few studies have examined their impact on real-world patient outcomes. Our objective was to assess the impact of a deep-learning model (COMPOSER) for the early prediction of sepsis on patient outcomes. We completed a before-and-after quasi-experimental study at two distinct Emergency Departments (EDs) within the UC San Diego Health System. We included 6217 adult septic patients from 1/1/2021 through 4/30/2023. The exposure tested was a nurse-facing Best Practice Advisory (BPA) triggered by COMPOSER. In-hospital mortality, sepsis bundle compliance, 72-h change in sequential organ failure assessment (SOFA) score following sepsis onset, ICU-free days, and the number of ICU encounters were evaluated in the pre-intervention period (705 days) and the post-intervention period (145 days). The causal impact analysis was performed using a Bayesian structural time-series approach with confounder adjustments to assess the significance of the exposure at the 95% confidence level. The deployment of COMPOSER was significantly associated with a 1.9% absolute reduction (17% relative decrease) in in-hospital sepsis mortality (95% CI, 0.3%-3.5%), a 5.0% absolute increase (10% relative increase) in sepsis bundle compliance (95% CI, 2.4%-8.0%), and a 4% (95% CI, 1.1%-7.1%) reduction in 72-h SOFA change after sepsis onset in causal inference analysis. This study suggests that the deployment of COMPOSER for early prediction of sepsis was associated with a significant reduction in mortality and a significant increase in sepsis bundle compliance.

Background

Objective and early identification of hospitalized patients, and particularly those with novel coronavirus disease 2019 (COVID-19), who may require mechanical ventilation (MV) may aid in delivering timely treatment.

Research question

Can a transparent deep learning (DL) model predict the need for MV in hospitalized patients and those with COVID-19 up to 24 h in advance?

Study design and methods

We trained and externally validated a transparent DL algorithm to predict the future need for MV in hospitalized patients, including those with COVID-19, using commonly available data in electronic health records. Additionally, commonly used clinical criteria (heart rate, oxygen saturation, respiratory rate, Fio₂, and pH) were used to assess future need for MV. Performance of the algorithm was evaluated using the area under receiver operating characteristic curve (AUC), sensitivity, specificity, and positive predictive value.

Results

We obtained data from more than 30,000 ICU patients (including more than 700 patients with COVID-19) from two academic medical centers. The performance of the model with a 24-h prediction horizon at the development and validation sites was comparable (AUC, 0.895 vs 0.882, respectively), providing significant improvement over traditional clinical criteria (P < .001). Prospective validation of the algorithm among patients with COVID-19 yielded AUCs in the range of 0.918 to 0.943.

Interpretation

A transparent deep learning algorithm improves on traditional clinical criteria to predict the need for MV in hospitalized patients, including in those with COVID-19. Such an algorithm may help clinicians to optimize timing of tracheal intubation, to allocate resources and staff better, and to improve patient care.