A Method for Grouping Emergency Department Visits by Severity and Complexity

Introduction Triage functions to quickly prioritize care and sort patients by anticipated resource needs. Despite widespread use of the Emergency Severity Index (ESI), there is still no universal standard for emergency department (ED) triage. Thus, it can be difficult to objectively assess national trends in ED acuity and resource requirements. We sought to derive an ESI from National Hospital Ambulatory Medical Care Survey (NHAMCS) survey items (NHAMCS-ESI) and to assess the performance of this index with respect to stratifying outcomes, including hospital admission, waiting times, and ED length of stay (LOS). Methods We used data from the 2010–2015 NHAMCS, to create a measure of ED visit complexity based on variables within NHAMCS. We used NHAMCS data on chief complaint, vitals, resources used, interventions, and pain level to group ED visits into five levels of acuity using a stepwise algorithm that mirrored ESI. In addition, we examined associations of NHAMCS-ESI with typical indicators of acuity such as waiting time, LOS, and disposition. The NHAMCS-ESI categorization was also compared against the “immediacy” variable across all of these outcomes. Visit counts used weighted scores to estimate national levels of ED visits. Results The NHAMCS ED visits represent an estimated 805,726,000 ED visits over this time period. NHAMCS-ESI categorized visits somewhat evenly, with most visits (42.5%) categorized as a level 3. The categorization pattern is distinct from that of the “immediacy” variable within NHAMCS. Of admitted patients, 89% were categorized as NHAMCS-ESI level 2–3. Median ED waiting times increased as NHAMCS-ESI levels decreased in acuity (from approximately 14 minutes to 25 minutes). Median LOS decreased as NHAMCS-ESI decreased from almost 200 minutes for level 1 patients to nearly 80 minutes for level 5 patients. Conclusion We derived an objective tool to measure an ED visit’s complexity and resource use. This tool can be validated and used to compare complexity of ED visits across hospitals and regions, and over time.


INTRODUCTION
In the current practice of emergency medicine, triage functions to quickly prioritize care and sort patients by anticipated resource needs. While the goal of accurate What was the research question? Derive an ESI tool from a national survey item and assess the performance of this index with respect to stratifying outcomes.
What was the major finding of the study? This tool can be used to compare complexity of ED visits across hospitals and regions, and over time.
How does this improve population health? ESI may not reflect resource needs in linear fashion. Our tool helps to compare data across regions and time periods.
important as the number of ED visits continues to rise, hospitals function under reduced available capacity, and reliance on the ED as the safety net of hospital systems increases. 1 This dual function of ED triage was proposed by Wuerz, who pioneered a five-level Emergency Severity Index (ESI). 2 Several studies have evaluated ESI's reliability and validity. [2][3][4][5][6][7][8][9][10] Other five-level triage schemes have been developed outside the United States: the Canadian Triage and Acuity Scale (CTAS), [11][12] and the Australasian Triage Scale (ATS). 13 Both CTAS and ATS assign each triage level a target "time to be seen," which in turn allows comparisons between sites on the basis of compliance with these targets. The ATS is unique in explicitly accepting a third role as a data source for describing case-mix to generate the adjusted estimates of ED visit characteristics that inform national policy. 12 Comparing ED performance and ED visit characteristics is more problematic in the US. Despite geographic variation in important ED characteristics (eg, the proportion of safetynet visits, 14 population-based ED visit rates, 15 and hospital admission rates 16 ) there is still no national mandated standard for categorizing the acuity and resource complexity of ED visits. Central to the ESI system is the idea of "immediacy," a marker of how acutely ill a patient is believed to be and thus how "immediately" they may need to be seen. Unfortunately, because of the continued widely disparate triage procedures and non-response in surveys, adjustment of ED data on the basis of the "immediacy" item alone is potentially biased and may ignore the dimension of care complexity. Additionally, the acuity or "immediacy" of a patient, as denoted by ESI, often does not linearly correlate with resource utilization during that ED visit.
The purpose of this study was to evaluate the "immediacy" variable in existing National Hospital Ambulatory Medical Care Survey (NHAMCS) survey data, and to create a practical alternative method for grouping ED visits by both acuity and resource complexity in a manner analogous to the ESI. To minimize data loss we sought to derive an ESI from NHAMCS survey items with low frequencies of item non-response. We assessed the performance of this index with respect to several outcomes, including hospital admission, waiting time, and overall ED length of stay (LOS).
This study uses the combined 2010-2015 ED components of the NHAMCS. 16 The NHAMCS is a probability sample of US hospital EDs and outpatient departments conducted annually since 1992. It is one of a family of healthcare surveys performed by the Centers for Disease Control and Prevention's (CDC) National Center for Health Statistics (NCHS). The US Census Bureau is responsible for field operations and data collection. Although one of its data items is currently a five-level item called "immediacy with which patient should be seen," with additional checkboxes for "no triage" and "unknown," other measures of urgency have been abstracted from ED charts in the past. From 1992-1996 the survey captured a highly subjective two-level "Urgent/emergent vs. Non-urgent" item, which led to the widely cited and heavily criticized conclusion that "55% of ED visits are non-urgent." 17 In 1997 this item was replaced by a four-level variable to capture more degrees of immediacy, each succeeding level associated with a progressively longer target "time to be seen." In 2005 "immediacy" was promoted to the current five-level item, each level again associated with target times.

METHODS
The NHAMCS is a four-stage probability sample, sampled in the following sequence: 1) 112 geographic primary sampling units of approximately county size; 2) probability sample of nonfederal, short-stay, general hospitals with EDs or outpatient departments or both, within the sampled primary sampling units, selected from a publicly available database of all US hospitals; 3) emergency service areas within 24-hour EDs and clinics within outpatient departments; and 4) a sample of about 100 visits within the selected EDs or outpatient departments during a randomly assigned four-week reporting period throughout the year. We limited our analysis to the ED component of NHAMCS and downloaded data from the NHAMCS website (ftp://ftp.cdc.gov/ pub/Health_Statistics/NCHS/Datasets/NHAMCS). Hospital staff were asked to complete a patient record form (PRF) for a sample of visits during a four-week reporting period, from which the data were abstracted and coded. The NHAMCS was approved by

Theiling et al.
A Method for Grouping ED Visits by Severity and Complexity Duke University Insitutational Review Board. A report published elsewhere describes the plan and operation of the NHAMCS in greater detail. 17 Unless otherwise noted, all estimates in this report are weighted to give national estimates. We considered estimates based on an unweighted count of less than 30 to be unreliable.

Creating The NHAMCS-ESI Index
We based the NHAMCS-ESI (ESI-N) on the published ESI, 2 but used only variables available in NHAMCS (Table  1). Since ESI and other tools are used in the initial triage process, they are dependent on data available immediately upon or shortly after presentation. Thus, ESI-N uses the presenting complaint rather than the final diagnosis as the main component. For NHAMCS-ED, this complaint is abstracted directly from the actual ED chart into up to three free-text entry fields on the PRF. The PRFs are then batched, and the handwritten text is converted to standard codes by the Constella Group, Inc. ( especially among the pediatric population; thus, when vital signs were missing, we considered the ESI-N unchanged rather than missing. Unlike the ESI, the retrospective ESI-N tallies actual resources used, rather than predicted resources, and cannot account for any resources not listed in the PRF.

Describing The Acuity Of The Patients And Validating The Index
We described the acuity of the study population by generating basic descriptive statistics including the mean ESI-N level and corresponding confidence intervals (CI) for selected patient and hospital characteristics. To validate the ESI-N, we examined associations with typical indicators of acuity such as waiting time, LOS, disposition, and mode of arrival. Waiting time was defined as the number of minutes between the time of arrival and the time seen by a physician. We defined LOS as the number of minutes between the time of arrival and time of ED discharge. All of these outcomes measures of acuity (waiting time, LOS, disposition, and mode of arrival) were measured in their respective units or proportions for each level of the derived ESI-N and compared using 95% CIs. We conducted all analyses using Stata v10 (StataCorp, College Station, TX). 19

RESULTS
The "immediacy" item in NHAMCS-ED (IMMED) was unknown, or triage was not performed, in 15.0% of visits. Among the remaining visits, the maximum value of IMMED was less than five in 145 (26.2%) of 553 emergency service areas in the 457 hospitals surveyed. This suggests the continued use of triage schemes with fewer than five levels. Unfortunately, having four or fewer triage levels creates a bias toward lower (more acute) immediacy levels when incorporated in aggregate estimates.
These results are given both in raw counts of 2010-2015 patient record forms, and in a nationally representative estimate of percent of all visits. Figure 1 compares this frequency distribution both to the distribution of the variable IMMED in the same data, and to the distribution of actual ESI levels assigned by triage nurses in a prospective validation study of two ED populations. 20 We assessed mean ESI levels for a variety of patient characteristics, as demonstrated in Table 2. For each of the four patient-level characteristics that we assessed (age, gender, race/ethnicity, and payer type), there were statistically significant differences (p<0.0001) between mean ESI-N levels. Of note, there was a monotonic increase in acuity with increasing age. Within their respective categories, visits by women and by non-Hispanic Whites had on average more acute ESI-N scores. Among payer types, Medicare visits had on average the most acute ESI-N, with the least acute categories being worker's compensation, "no charge," and Medicaid/State Children's Hospital Insurance Program.
We assessed validity of the ESI-N against several

Theiling et al.
A Method for Grouping ED Visits by Severity and Complexity outcomes, including hospital admission, intensive care unit (ICU) admission, ambulance arrival, and leaving without being seen and in each case found the expected relationships ( Table 3). As can be seen, the ESI-N is able to differentiate all visits into five levels of care, with only a minority being classified in the most severe, "Immediate" category. Relative to their proportion of total visits, more severe acuity ESI-N levels account for a higher proportion of the patients admitted to the hospital and those admitted to a critical care unit. Furthermore, ESI-N is able to differentiate visits ( Table  4) by likelihood of being admitted to the hospital and ED LOS. The 95% CIs of percent admitted or LOS in minutes between categories largely do not overlap as the ESI-N increases in severity from Level 5 "Nonurgent" to Level 1 "Immediate." ESI-N does not appear to differentiate well among categories for time waiting to be seen by a physician, however.

DISCUSSION
The NHAMCS-ED is the only nationally representative survey of ED visits; thus, it is a valuable resource for policymaking and has provided data for many published studies. However, most causal inferences of interest, such as the effect of demographic variables like race/ethnicity on hospital admission, length of ED visit, and other aspects of quality and cost, are potentially confounded by the main determinant of these outcomes: the seriousness or acuity of the patient's initial problem. A related reason to measure acuity is the need for fair comparisons across regions or over time, i.e., for case-mix adjustment. We have derived a five-level index of acuity called the ESI-N, because a similar existing item in the survey, called "immediacy," frequently offered difficulties in chart abstraction: either a triage score was not obtained, or, in some cases when the score was obtained, it was absent from the chart. In addition, even when a triage score was present, there still is no universally accepted standard of triage in the US. In many cases, EDs may use fewer than five triage levels; in others the identical levels in different systems may have different definitions or applications. We attempted to overcome these limitations by deriving ESI-N, a severity index based primarily on the patient's RFV codes, modified by values in a small number of other data fields, including age, vital sign extremes, and resuscitative procedures. Finally, we separated the less acute visits by number of resources used, emulating the previously derived and validated ESI. 2 The American College of Emergency Physicians (ACEP) and the Emergency Nurses Association (ENA) formed a task force in 2002 dedicated to the goal of creating and promoting a standard measure of presenting patient acuity. 21 However, to this day, a number of EDs continue to use locally designed triage guidelines of varying complexity and evidence quality or do not perform formal triage. 22 In fact, this continued lack of a standard resulted in the ACEP and ENA revising its position statement and advocating for "implementing a standardized emergency department (ED) triage scale and acuity categorization process" and endorsing ESI as that process. 23 While ESI remains a prominent triage tool, it does not always adequately reflect resource need in linear fashion. That is, ESI Level 1s do not always necessitate the greatest

A Method for Grouping ED Visits by Severity and Complexity
Theiling et al.
resources in the ED, with ESI 2s requiring less, and so on. In fact, one paper found that ESI Level 2 and 3 patients are actually very similar in their resource needs, but hospitalization varied dramatically. 24 In contrast, we found positive associations between ESI-N severity and hospital admission, ICU admission, and ambulance transport. 2 Unlike one prominent early study, 22 we found that patients who left without being seen were much more likely to have a low

Theiling et al.
A Method for Grouping ED Visits by Severity and Complexity severity by ESI-N, a change that could reflect increased use of the ED as a "safety net" rather than for emergencies. This is congruent to findings of some more recent studies. [25][26] Waiting time is often cited as an indicator of ED quality that depends both on hospital capacity as well as the average ED volume and acuity of visits. Recent studies suggest that ED crowding has increased waiting times even for serious problems. 27 But excluding triage category 1, the ESI-N index discriminates poorly, possibly in part because the nurse assigning a triage category also controls the patient's waiting time.

LIMITATIONS
While the ESI-N derived here appears to be a consistent, valid measure of acuity and complexity, there are limits on its use. It is complex: Creating the index requires an algorithm executed by a computer program. It is derived from the 2010-2015 NHAMCS-ED survey files, which were formatted in a specific way; however, variable names and values, such as the RVF classification scheme, have changed and will continue to change over time. Use of the algorithm in survey years other than 2010-2015 may require its modification. Like the ESI itself, devising the ESI-N algorithm required subjective judgment.
The index reflects two separate dimensions of ED visits: acuity and complexity. Distinguishing between these dimensions is impossible when evaluating aggregate data. In this paper we describe and validate the ESI-N. It has not been validated and tested for reliability.

CONCLUSION
Two technical tasks commonly required in health services research include accounting for confounding of a causal relationship, and adjusting for case-mix to minimize selection bias when comparing groups. 28 Our derivation of a five-level severity index for data abstracted from ED charts addresses these research needs. The ESI-N can be used to stratify results, or as an ordinal exposure or outcome variable in regression or propensity score models, increasing statistical power by reducing the need to include multiple covariates in a model. Caution should be exercised in its application. While it includes observations lost with the use of IMMED, it is less predictive of ED waiting time. It might be used together with IMMED to reduce residual confounding in some analyses. It is important to understand the origin of the index in complaint codes modified by age and a few other variables, and that it segregates less acute  ESI-N, Emergency Severity Index levels based primarily on the patient's "reason for visit" code as presented in the National Hospital Ambulatory Care Survey.
A Method for Grouping ED Visits by Severity and Complexity Theiling et al.
visits based on a list of specified services performed. Since this list is short, and since some of the data abstracted are relatively nonspecific (eg, five-level instead of 10-level pain scores), the ESI-N will misclassify some patients compared to a prospectively determined ESI. Future directions include validating ESI-N on another independent source of ED visits and testing its use in a prospective cohort of ED visits.