The Accuracy of Sepsis Screening Score for Mortality Prediction at Emergency Department Triage

Introduction Sepsis has a mortality rate of 10–40% worldwide. Many screening tools for sepsis prediction and for emergency department (ED) triage are controversial. This study compared the accuracy of the scores for predicting 28-day mortality in adult patients with sepsis in the triage area of the ED. Methods Adult patients who presented to the ED of a tertiary-care university hospital from January–December 2019 with an initial diagnosis of sepsis or other infection-related conditions were enrolled. We calculated predictive scores using information collected in the ED triage area. Prognostic accuracy was measured by the area under the receiver operating characteristic curve (AUROC) for predicting 28-day mortality as a primary outcome. The secondary outcomes included mechanical ventilation usage and vasopressor usage for 28 days. Results We analyzed a total of 550 patients. The 28-day mortality rate was 12.4% (n = 68). The 28-day mortality rate was best detected by the National Early Warning Score (NEWS) (AUROC = 0.770; 95% confidence interval [CI]: 0.705–0.835), followed by the quick Sequential Organ Failure Assessment (qSOFA) score (AUROC = 0.7473; 95% CI: 0.688–0.806), Search Out Severity (SOS) score (AUROC = 0.749; 95% CI: 0.685–0.815), Emergency Severity Index (ESI) triage (AUROC = 0.599; 95% CI: 0.542–0.656, and the Systemic Inflammatory Response System (SIRS) criteria (AUROC = 0.588; 95% CI: 0.522–0.654]). The NEWS also provided a higher AUROC and outperformed for 28-day mechanical ventilator usage and 28-day vasopressor usage. Conclusion The NEWS outperforms qSOFA, SOS, SIRS, and ESI triage in predicting 28-day mortality, mechanical ventilator, and vasopressor usage of a patient with sepsis who is seen at ED triage.


INTRODUCTION
Sepsis is a clinical syndrome of life-threatening organ dysfunction caused by a dysregulated host response to infection. 1 Over the past 30 years, sepsis has increasingly become an area of interest both in diagnosis and management because of its high mortality rate. Despite this increased focus, the mortality rate of sepsis is still high, 2 averaging 39% worldwide. 3 The Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3) recommended the application of the Sequential Organ Failure Assessment (SOFA) to identify organ dysfunction or failure in sepsis patients. 1 When SOFA was compared with the original Systemic Inflammatory Response Syndrome (SIRS) criteria, SOFA outperformed SIRS in predicting hospital mortality. The consensus suggested quick sequential organ failure assessment (qSOFA) as a screening tool in patients who are likely to have sepsis; qSOFA was proven to offer predictive validity similar to SOFA. 4

Suttapanit et al.
Accuracy of Sepsis Screening Score for Mortality Prediction at ED Triage

Population Health Research Capsule
What do we already know about this issue? Many screening tools are available at the triage area of the emergency department.
What was the research question? Which triage screening tool is the most accurate for predicting mortality in patients with sepsis?
What was the major finding of the study?
The National Early Warning Score outperforms other sepsis screening tools (area under the receiver operator characteristic curve of 0.77) in predicting mortality, need for ventilator and vasopressors for patients evaluated at ED triage.
How does this improve population health?
Using the most accurate screening tool at ED triage could enhance the healthcare of the population, including patients with sepsis.
In 2016 the Surviving Sepsis campaign recommended the implementation of sepsis screening, which has been shown to improve outcomes and reduce the mortality rate. 5 Many predictive scores, such as the National Early Warning Score (NEWS), were developed and implemented to detect deterioration in sepsis patients. 4 These scores can be used as a general screening tool as well as an early warning tool in the emergency department (ED), guiding collaboration with other areas in the hospital and the patient care system. The Emergency Severity Index (ESI) triage tool is a five-level ED triage algorithm that provides clinically relevant stratification of patients from 1 (the most emergent priority) to 5 (the least urgent priority) based on acuity and resource needs. 6 . However, the ESI triage tool was not specifically designed for severity classification in sepsis patients. The Search Out Severity (SOS) score was the early sepsis score used in Thailand. It has been shown that the implementation of a combined SOS score for screening with a checklist for sepsis bundles could decrease the mortality rate in Thailand. 2 This study compares the accuracy of qSOFA, NEWS, SOS, SIRS, and ESI triage for predicting 28-day mortality in adult patients with sepsis, with the goal of designing an appropriate screening tool for use in the ED triage area.

Study Design and Setting
This was a retrospective cross-sectional study. We collected data in the ED of a tertiary-care university hospital, between January-December 2019. The study was approved by the Ethics Committee of our institution.

Study Population
The study included patients >18 years who presented to the ED with a diagnosis of sepsis or infection-related conditions (Appendix 1) and had been treated with the sepsis protocol in the ED. We enrolled patients by day and alternated the days of the ED visit to reach the calculated sample size. The exclusion criteria were patients who transferred from other hospitals or areas of the hospital and patients with incomplete 28-day follow-up data.

Data Measurement and Outcomes
Data collection included patient demographics, presenting symptoms, vital signs recorded at the triage area, provisional diagnosis, hemoculture status, site of infection, 28-day intubation status, 28-day vasopressor time, and 28-day mortality. The variables of qSOFA, NEWS, SOS, SIRS, and ESI triage were recorded using the information gathered from the triage area of the ED (Appendix 2). The primary outcome was 28-day mortality. The secondary outcomes were mechanical ventilator usage within 28 days and vasopressor usage within 28 days.
Suspected sepsis was defined by physicians in the ED using the sepsis protocol, including qSOFA in Sepsis-3 criteria 1 or physicians' clinical judgment in the ED. Some physiologic parameters were not used because our goal was to compare predictive scores, which were used as a screening tool in the ED triage area. Thus, for example, the maximum score for SIRS was 3 because white blood cell count was disregarded, and the SOS score did not include urine output. Furthermore, a Barthel index of 20 was used to define totally dependent activities of daily living (ADL), and heart failure with reduced ejection function (HFrEF) was defined as a left ventricular ejection fraction of 40% on transthoracic echocardiography, which was documented in the medical records.

Sample Size and Data Analysis
We calculated the sample size for this study by using the equation N = Z α/2 2 p(1 − p)/d 2 , with the standard normal variate (Z α/2 ) at 5%, the probability of expected sensitivity (p) equals 0.9. A two-sided test concluded that the minimum sample size would be 139 samples. The mortality rate for sepsis is 39%, as reported in a previous study. 3

Statistical Analysis
We compared the survival and the nonsurvival groups by using the chi-square or Fisher's exact test for categorical variables and the t-test for continuous variables. The data was presented as a percentage for categorical data and as a mean with standard deviation or median with interquartile range, as appropriate, for numerical data. The area under the receiver operating characteristic curve (AUROC), with a 95% confidence interval (CI), was depicted to evaluate the discrimination performance of each score. Sensitivity Accuracy of Sepsis Screening Score for Mortality Prediction at ED Triage Suttapanit et al.
and specificity were calculated for each score as well. A P-value less than 0.05 was considered significant. We used STATA version 16.1 for statistical analysis (StataCorp LLC, College Station, TX).

RESULTS
In total, 550 patients were included in the analysis. A protocol flow chart is shown in Figure 1. The overall 28day mortality was 12.4%. The overall 28-day mechanical ventilator usage and 28-day vasopressor usage were 23.2% and 18.1%, respectively. The mean age was 69 years, and 46.7% of patients were male. The three most common comorbidities were diabetes mellitus (31.6%), solid-organ malignancy (25.8%), and totally dependent ADL (19.8%). The mortality was significantly higher in comorbidities such as the solid organ tumor group, the hematologic malignancy group, and HFrEF group. Vital signs such as higher heart rate (118 vs 106, P <0.001) and respiratory rate (27 vs 24), P <0.001) and lower systolic blood pressure (112 vs 106, P <0.001) and oxygen saturation (92 vs 96), P <0.014) were significant in mortality. The patient demographic data in the survival and nonsurvival groups, is summarized in Table 1.
The primary outcome, 28-day mortality, was best detected by NEWS (area under the receiver operating characteristic curve [AUROC] = 0.770; 95% CI: 0.705-0.835), followed by SOS (AUROC = 0.749; 95% CI: 0.685-0.815, qSOFA  Table 2 and Figure 2. The sensitivity and specificity for predicting the 28-day mortality rates of all predictive scores at different threshold are presented in Table 3.
For the secondary outcomes, 28-day mechanical ventilator usage and vasopressor usage, NEWS provided a high AUROC and outperformed as shown in Table 2

DISCUSSION
Our study demonstrates that NEWS has the best predictive performance for the 28-day mortality of sepsis patients at the triage area of the ED. In the same way, Omar et al reported that NEWS outperformed both SIRS (AUROC 0.95 vs 0.89; P 0.001) and qSOFA (AUROC 0.95 vs 0.87; P 0.001) in predicting death in only the severe sepsis and septic shock groups in the ED. 7 Anniek et al determined that NEWS performed substantially better than qSOFA and SIRS in predicting both 10-day mortality (AUROC = 0.837, 0.744, and 0.646, respectively) and 30-day mortality (AUROC = 0.779, 0.697, and 0.631, respectively). 8 Furthermore, NEWS showed a high performance in predicting 28-day mechanical ventilator and 28-day vasopressor used. These results were in accordance    with the previous study by Churpek et al, 9 which showed that general early warning scores (EWS) are more accurate than qSOFA in predicting adverse outcomes of sepsis outside the intensive care unit setting.
In the triage area of the ED, qSOFA was easier to assess by less experienced medical professionals. 1 However, qSOFA has a limited ability to predict poor outcomes in sepsis patients. 10,11 Additionally, the metrics used in EWS are standard measures that can be readily and rapidly performed throughout the healthcare system as well as in the ED triage area. The NEWS also demonstrates a higher performance than the SOS score, which necessitates information not available in the triage area. Chompunot et al 2 conducted a study in the hospital referral system that did not focus on the triage area. Their study found that the ESI score, which is commonly used as the general screening tool at ED triage,   was inferior to NEWS, SOS, and qSOFA in predicting sepsis-related 28-day mortality, 28-day mechanical ventilator, and 28-day vasopressor used. Moreover, determining ESI at triage requires evaluator experience, as well as differing cut-off values of the parameters with other tools from other patient care systems.
Because of its strong predictive accuracy and simplicity, our findings support the use of NEWS as a screening tool in ED triage. 5,[12][13][14][15] An automatic calculation in the sepsis alert system likewise correctly uses NEWS. 16 A NEWS cut-off prediction score of > 4 (sensitivity 91.30%, specificity 28.69%) and > 5 (  specificity 45.74%) predicted sepsis-related 28-day mortality, according to our findings. The score has the highest sensitivity (90%) and specificity (25%) for activating sepsis alarms.

LIMITATIONS
This was a retrospective, single-center study. Second, it should be noted that substantial numbers of patients had advanced-stage malignancies, including solid organ and hematologic malignancies, which had a higher mortality rate. Additionally, patients who did not resuscitate were not excluded from our study, which could have affected the outcome.

CONCLUSION
The National Early Warning Score outperforms qSOFA, SOS, SIRS, and ESI triage scores in predicting 28-day mortality, mechanical ventilator usage, and vasopressor usage of a patient with sepsis in the triage area of the ED.