Prognostic Accuracy of SpO2-based Respiratory Sequential Organ Failure Assessment for Predicting In-hospital Mortality

Introduction In this study we aimed to investigate the prognostic accuracy for predicting in-hospital mortality using respiratory Sequential Organ Failure Assessment (SOFA) scores by the conventional method of missing-value imputation with normal partial pressure of oxygen (PaO2)- and oxygen saturation (SpO2)-based estimation methods. Methods This was a single-center, retrospective cohort study of patients with suspected infection in the emergency department. The primary outcome was in-hospital mortality. We compared the area under the receiver operating characteristics curve (AUROC) and calibration results of the conventional method (normal value imputation for missing PaO2) and six SpO2-based methods: using methods A, B, PaO2 is estimated by dividing SpO2 by a scale; with methods C and D, PaO2 was estimated by a mathematical model from a previous study; with methods E, F, respiratory SOFA scores was estimated by SpO2 thresholds and respiratory support use; with methods A, C, E are SpO2-based estimation for all PaO2 values, while methods B, D, F use such estimation only for missing PaO2 values. Results Among the 15,119 patients included in the study, the in-hospital mortality rate was 4.9%. The missing PaO2was 56.0%. The calibration plots were similar among all methods. Each method yielded AUROCs that ranged from 0.735–0.772. The AUROC for the conventional method was 0.755 (95% confidence interval [CI] 0.736–0.773). The AUROC for method C (0.772; 95% CI 0.754–0.790) was higher than that of the conventional method, which was an SpO2-based estimation for all PaO2 values. The AUROC for total SOFA score from method E (0.815; 95% CI 0.800–0.831) was higher than that from the conventional method (0.806; 95% CI 0.790–0.822), in which respiratory SOFA was calculated by the predefined SpO2 cut-offs and oxygen support. Conclusion In non-ICU settings, respiratory SOFA scores estimated by SpO2 might have acceptable prognostic accuracy for predicting in-hospital mortality. Our results suggest that SpO2-based respiratory SOFA score calculation might be an alternative for evaluating respiratory organ failure in the ED and clinical research settings.


INTRODUCTION
4][5][6] The most recent revision of the sepsis definition (Sepsis-3) stresses the defining feature of sepsis as a "dysregulated host response to infection" and emphasizes focus on quantification of organ dysfunction. 1,7The Sepsis-3 definition adopts the Sequential Organ Failure Assessment (SOFA) score as a measure of organ failure, and the clinical criteria of sepsis included acute change in SOFA score. 7,8hile various scoring systems can be used for prognostication of suspected sepsis patients, the SOFA score is the most validated system and an essential component of a clinical sepsis definition. 9The SOFA score was initially designed to provide population-level insights into acute morbidity in intensive care unit (ICU) patients, but it has become integrated into many aspects of critical care in both ICU and non-ICU settings including the ED. 10 The SOFA score is based on six organ categories, one for each of the respiratory, cardiovascular, hepatic, coagulation, renal, and neurological systems, each scored from 0 to 4, with an increasing score reflecting worsening organ dysfunction. 11he severity of respiratory dysfunction is measured with the SOFA score based on the ratio of partial pressure of oxygen (PaO 2 ) to fraction of inspired oxygen (FiO 2 ) (PF).The PF ratio provides information about pulmonary gas exchange adjusted for the quantity of oxygen delivered. 12lthough PaO 2 is a reference variable, invasive arterial blood gas (ABG) measurements are infrequently performed, and PF ratios are often unavailable for patients outside the ICU. 1 Furthermore, PaO 2 is often measured once rather than multiple times, which reduces clinical utility in non-ICU settings.In clinical studies, missing PaO 2 values are usually considered normal.As a noninvasive alternative to PaO 2 , peripheral oxygen saturation (SpO 2 )-based estimation and the SpO 2 /FiO 2 (SF) ratio have been proposed, but comparative data of estimation methods including simplified or mathematical models in non-ICU settings are limited and require further validation. 12n this study we aimed to investigate the prognostic accuracy for predicting in-hospital mortality of respiratory SOFA scores by the conventional method of missing value imputation with normal PaO 2 -and SpO 2 -based estimation methods.

Study Design
This was a single-center, retrospective cohort study of patients with suspected infection who presented to the ED of a tertiary-care hospital located in a metropolitan city between December 2017-November 2019.This study was approved by the Institutional Review Board of Samsung Medical Center (No. SMC 2022-08-158-001).The requirement for informed consent was waived given the study's retrospective nature and anonymized patient data.We followed the guidelines of the Strengthening the Reporting of Observational Studies in Epidemiology Statement (Appendix 1).

Study Population and Definitions
We included patients ≥18 years old with suspected infection who presented to the ED.Suspected infection was defined as cases in which blood culture and antibiotic administration were conducted in the ED. 1,13We excluded patients who had limitations on invasive care (eg, patients who had terminal malignancy or who had previously signed a do-not-resuscitate [DNR] order), who presented with cardiac arrest, who had obvious non-infectious conditions such as trauma or bleeding, who were without SpO 2 or FiO 2 , or had inadequate data due to our inability to access their electronic health record (EHR).

Data Collection and Outcome Measurements
We collected retrospective cohort data by extraction from the hospital's clinical data warehouse and review of EHR.

Population Health Research Capsule
What do we already know about this issue?Although PaO 2 is a reference value in the Sequential Organ Failure Assessment (SOFA) score, it is often unavailable for non-ICU patients.
What was the research question?Are respiratory SOFA scores estimated by SpO 2 comparable to the conventional method for predicting in-hospital mortality?
What was the major quantitative finding of the study?
The AUROC of the SpO 2 -based respiratory SOFA (0.772; 95% CI 0.754-0.790)was higher than that of the conventional method.

How does this improve population health?
Respiratory SOFA scores estimated by SpO 2 might be an alternative way to evaluate respiratory organ failure in the emergency department and clinical research.
Eligible cases were electronically identified by the aforementioned definition.Data extraction was carried out by two designated research coordinators trained on the definition of each variable by the investigator and who were blinded to the study hypothesis.To ensure high quality, one investigator reviewed the EHRs and verified the final data to resolve data conflicts.The following data were retrieved: demographic characteristics including age and gender; comorbidities; vital signs; laboratory data including platelet count, bilirubin, creatinine, lactate, and ABG analysis; vasopressor use; SOFA score; FiO 2 and mechanical ventilation support; infection focus; and outcome-related data including in-hospital mortality and 28-day mortality.For collecting mortality data, we used visit history after discharge, mortality data provided by Statistics Korea, and telephone interviews.The primary endpoint was in-hospital mortality.

Respiratory SOFA Score Assessment
Detailed equations for assessing respiratory SOFA score are shown in Table 1.As a conventional method, we calculated respiratory SOFA by PaO 2 value and imputation as a normal value for missing PaO 2 .We used estimated PaO 2 values from SpO 2 based on two previously suggested methods (from Madan et al and Sauthier et al). 14,15We replaced all PaO 2 (methods A and C) with estimated values regardless of the presence of measured PaO 2 , or we imputed missing PaO 2 with estimated values (methods B and D).We also estimated respiratory SOFA scores by SpO 2 and respiratory support use in all cases (method E) or in cases with missing PaO 2 values (method F).We used a modified model from Valik et al because the original study did not incorporate use of respiratory support. 16All SOFA score components were calculated using maximum values during the 24 hours after ED arrival.Estimation of FiO 2 in patients receiving supplementary oxygen is shown in Table S1.

Statistical Analyses
Results are presented as median values with interquartile ranges (IQR) for continuous variables and numbers of patients with percentages for categorical data.Continuous and categorical variables were analyzed by the Kruskal-Wallis test and chi-square test, respectively.We compared prognostic performance of estimated respiratory SOFA score from each method with conventional respiratory SOFA score calculation for predicting in-hospital mortality.The estimated total SOFA scores from estimation methods for respiratory SOFA were compared to the total SOFA score by the conventional method.Discrimination was measured using the area under the receiver operating characteristic curve (AUROC).We also calculated the exact binominal 95% confidence interval (CI) for the AUROC.We measured the differences between conventional respiratory SOFA score AUROC and estimated respiratory SOFA score AUROC using the method proposed by DeLong et al. 17 Calibration was assessed using calibration plots based on 100 bootstrap replicates.A P-value less than 0.05 was considered significant.We used R version 4.1.3(R Foundation for Statistical Computing, Vienna, Austria; http://www.R-project.org/) for statistical analysis.

Study Population
We assessed the eligibility of 17,736 adult patients who underwent blood culture and antibiotic administration in the ED from December 2017-November 2019.After excluding patients who had limitations on invasive care (eg, patients who had terminal malignancy or who had previously signed a DNR order), presented with cardiac arrest, had obvious noninfectious conditions such as trauma or bleeding, were missing data on SpO 2 or FiO 2 , or had inadequate data due to inability to access the EHR, 15,119 patients were included in the analyses (Figure 1).As shown in Table 2, the overall median age was 63 years, and 8,248 of patients (54.6%) were male.Respiratory tract infection was the most common diagnosis, found in 4,523 patients (29.9%).The median PF ratio was 324.3 (IQR 255.2-388.1).The proportion of patients with missing PF ratio was 56.0%, and patients with data on PF ratio had higher in-hospital mortality (9.3% vs 1.4%; Table S2).The median SF ratio was 452.4 (IQR 443.0-461.9).Overall, the total conventional SOFA score was 2.0 (IQR 1.0, 4.0), and in-hospital mortality was 740 patients (4.9%).

Calibration of Respiratory SOFA Scores
Incidence and in-hospital mortality according to respiratory SOFA scores by the conventional method and the six estimation methods are shown in Figure 2. In-hospital mortality increased as estimated respiratory SOFA score increased in all methods.The calibration curve for inhospital mortality showed similar calibration for all methods (Figure S1).

Discrimination of Respiratory and Total SOFA Scores
The AUROCs of respiratory SOFA scores for predicting in-hospital mortality by the conventional method and by the six estimation methods are shown in Table 3 and Figure S2.The AUROC for method C (0.772; 95% CI 0.754-0.790)was significantly higher than that of the conventional method (0.755; 95% CI 0.736-0.773).The AUROCs of method B (0.739; 95% CI 0.719-0.759)and method D (0.735; 95% CI 0.715-0.755)were lower than that of the conventional method.The AUROCs of methods A (0.760; 95% CI 0.741-0.779),E (0.761; 95% CI 0.742-0.780),and F (0.758; 95% CI 0.739-0.777)were not significantly different from that of the conventional method.
The AUROCs for total SOFA scores for predicting in-hospital mortality are shown in Table 4.The AUROC for total SOFA score from method E (0.815; 95% CI 0.800-0.831)was statistically higher than that for the conventional method (0.806; 95% CI 0.790-0.822).The AUROCs for methods B and D were lower than that of the conventional method.The AUROCs for methods A, C, and F were similar to that of the conventional method.

DISCUSSION
In this single-ED study of 15,119 patients with suspected infection, PaO 2 values were commonly missing.Compared with a conventional missing value imputation with normal PaO 2 , SpO 2 -based estimation methods for missing PaO 2 did not improve the prognostic accuracy for predicting inhospital mortality.In contrast, respiratory SOFA scores estimated by SpO 2 , instead of measured and missing PaO 2 , yielded higher discrimination for respiratory SOFA assessment (method C using the equation from Sauthier et al)

<0.01
Bone or soft tissue 986 (  or total SOFA assessment (method E using a modified model from Valik et al).Our study showed that respiratory function assessment based on estimated respiratory SOFA scores from SpO 2 is comparable to the conventional scoring system and could facilitate respiratory dysfunction assessment in the ED.Our study is important because we included patients with suspected infection in a non-ICU setting, where PaO 2 measurement is limited but acute management of sepsis and septic shock usually take place.The SOFA score is a validated tool for organ failure assessment and for defining clinical sepsis. 1,7The association of SOFA score with clinical outcomes has led many investigators to propose it as a potentially valid surrogate in clinical trials. 3,9However, accurate respiratory SOFA score evaluation requires an invasive ABG measurement, which is not routinely ordered in patients outside the ICU due to limited resources and substantial risk of failure or complications. 3Jakobsen et al and Gadrey et al addressed the issue that multiple imputations of large proportions of missing data lead to unreliable outcomes. 18,19SpO 2 measured by pulse oximetry is a non-invasive, surrogate marker for tissue oxygenation that is routinely applied to most ED patients, and it can be monitored continuously. 20,21Previous studies introduced methods for imputing PaO 2 from SpO 2 .Rice et al found that the SF ratio correlates with a simultaneously obtained PF ratio in acute respiratory distress syndrome. 22authier et al developed and validated a method to filter SpO 2 streams to estimate PaO 2 using only continuous and noninvasive data. 15Valik et al showed that discrimination of mortality causes using SOFA score with respiratory function assessment based on SpO 2 is comparable with that of conventional respiratory function assessment. 16ll six estimated methods in our study replaced PaO 2 regardless of the presence of measured PaO 2 and yielded higher AUROCs for predicting in-hospital mortality.It is unclear why replacement of all PaO 2 values with estimated SpO 2 yielded better mortality-discriminant power than imputation of only missing PaO 2 values.It may be because it is difficult to perform ABG sequentially in the ED.As it suggests, sequential increases in SOFA score are associated with organ dysfunction. 23election of the lowest SpO 2 values from continuous monitoring might reflect deterioration in respiratory function better than does one-time PaO 2 measurement.SpO 2 measurement could identify more high-risk patients, including less severe patients, in the absence of PaO 2 values (Table S2).An optimal strategy or equation to assess respiratory SOFA score can be selected considering the clinical settings, severity of patients, and number of PaO 2 measurements.For example, we suggest that a simplified equation might be useful in resource-limited, urgent clinical settings like EDs.Among the six methods, Method E might be a good option for use in an ED.For clinical research, Method C would be preferred to show detailed data about estimated PaO 2 and betted discrimination performance of respiratory SOFA score.

LIMITATIONS
This study has several limitations.First, this was a singlecenter study conducted in the ED.Second, we were unable to assess pulse oximetry accuracy.There was the possibility that patient factors, such as skin pigmentation and peripheral circulation, affected SpO 2 measurement.Third, there might have been a selection bias in acquiring ABG measurements.For generalizability, further studies including representative patients in non-ICU settings are needed to determine the proper relationship between PaO 2 and SpO 2 .

CONCLUSION
Our study shows that respiratory SOFA scores estimated by SpO 2 might have acceptable or higher prognostic

Figure 2 .
Figure 2. Distribution and in-hospital mortality according to respiratory SOFA scores by the conventional method and six estimation methods.Bar graphs represent number of patients, and points with error bars indicate in-hospital mortality with 95% confidence interval: (A) Conventional respiratory SOFA score.(B) Estimated respiratory SOFA score from method A. (C) Estimated respiratory SOFA score from method B. (D) Estimated respiratory SOFA score from method C. (E) Estimated respiratory SOFA score from method D. (F) Estimated respiratory SOFA score from method E. (G) Estimated respiratory SOFA score from method F. SOFA, Sequential Organ Failure Assessment.
SOFA, Sequential Organ Failure Assessment; PaO 2 , partial pressure of oxygen in arterial blood; SpO 2 , peripheral oxygen saturation; mm Hg, millimeters of mercury.

Table 2 .
Baseline characteristics.The data are presented as median[IQR]for continuous variables or as number (%) for categorical variables.

Table 3 .
Area under the receiver operating characteristic curve for respiratory SOFA* scores for predicting in-hospital mortality by the conventional method and six estimation methods.*Conventional method respiratory SOFA score vs estimated respiratory SOFA score.