Where Have All the FLOWERS Gone? A Multicenter Investigation of Frequent Users of Midwest Emergency Department Services During the COVID-19 Stay-at-home Orders

Introduction In this study we aimed to determine the impact of the mandatory coronavirus disease 2019 (COVID-19) pandemic stay-at-home order on the proportional makeup of emergency department (ED) visits by frequent users and super users. Methods We conducted a secondary analysis of existing data using a multisite review of the medical records of 280,053 patients to measure the impact of the COVID-19 pandemic stay-at-home order on ED visits. The primary outcomes included analysis before and during the lockdown in determining ED use and unique characteristics of non-frequent, frequent, and super users of emergency services. Results During the mandatory COVID-19 stay-at-home order (lockdown), the percentage of frequent users increased from 7.8% (pre-lockdown) to 21.8%. Super users increased from 0.7% to 4.7%, while non-frequent users dropped from 91.5% to 73.4%. Frequent users comprised 23.7% of all visits (4% increase), while super user encounters (4.7%) increased by 53%. Patients who used Medicaid and Medicare increased by 39.3% and 4.6%, respectively, while those who were uninsured increased ED use by 190.3% during the lockdown. Conclusion When barriers to accessing healthcare are implemented as part of a broader measure to reduce the spread of an infectious agent, individuals reliant on these services are more likely to seek out the ED for their medical needs. Policymakers considering future pandemic planning should consider this finding to ensure that vital healthcare resources are allocated appropriately.


INTRODUCTION Background
On March 13, 2020, the United States issued a National Emergency Declaration to slow the spread of the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), the causative agent of the infectious coronavirus disease 2019 . Most state governors executed this declaration by implementing stay-at-home orders designed to limit people's movement in public and reduce viral spread. 1 These orders, directly and indirectly, resulted in a 42% reduction in emergency department (ED) visits nationally for a broad range of medical conditions and patient concerns. [2][3][4][5][6] What is unknown about the lockdown is whether the decrease in ED visits was uniform across all patient demographics or whether there were specific subgroups, such as frequent ED users (FEDU), whose habits deviated from this trend. Frequent ED users are patients who historically consume a significant percentage of acute care resources. 7,8 In general, these individuals have four or more ED visits per year, are more likely to suffer from three or more chronic medical conditions, have a higher incidence of mental health problems and substance use disorder, and account for a disproportionate amount of healthcare costs. [9][10][11][12] The FEDU tend to be socioeconomically disadvantaged and have higher usage rates of outpatient offerings (eg, social work services, addiction treatment, psychiatric counseling) than non-frequent ED users (NFEDU). 10,11 Overall, persons who seek out acute care more than four times per year represent between 3.5-29% of all ED patients but constitute 12.1-67% of all ED visits made. 12 A subset of frequent users visits the ED 10 or more times per year. These ED super users (EDSU) account for only 2.6-6.1% of all ED patients but comprise 16.2% of Medicare patients (≥65 years), 26.2% of Medicare patients (age 1 to 64 years), 16.7% of Medicaid patients, and 10.5% of those patients with private insurance. Only 3.7% of all Medicaid ED patients were super users, but they accounted for more than five times the average ED charge. 12 The frequent utilization of midwest emergency room services (FLOWERS) study is a retrospective analysis of the effects of the stay-at-home order on the use of ED services by FEDU and EDSU during the early phase of the COVID-19 pandemic. This investigation included 20 EDs with diverse demographics, economic bases, and hospital types (eg, tertiary referral hospitals, trauma centers, academic and community hospitals, and freestanding EDs).

Importance
Barriers to accessing healthcare services more commonly affect the impoverished, children, those with chronic illnesses, immigrants, the uninsured, and those with psychiatric and substance abuse disorders. Such barriers often lead to poorer health outcomes. 13 When access to primary medical care is limited or reduced, patients commonly seek out services in the ED. 14,15 The COVID-19 stay-at-home order was an emergency public health measure implemented in Ohio to help reduce community spread of disease during the pandemic. This measure created a broad, temporary barrier to healthcare access and a unique opportunity to assess its impact on at-risk patients who frequently use the ED for their healthcare needs.
The COVID-19 pandemic resulted in a marked drop in ED visits throughout the US and most of the world. 2 This reduction occurred against a backdrop of rising morbidity and mortality from COVID-19 and untreated medical emergencies. How individuals choose to address their medical needs and concerns during a government-imposed lockdown has broad implications for the healthcare market and mode of medical service delivery. It also influences the focus of emergency management and public health planning, resource allocation, and community support assistance in the future.

Goals of This Investigation
Our primary goal in this investigation was to determine the impact of the COVID-19 pandemic stay-at-home order on use of the ED by people who were FEDUs and EDSUs prior to the pandemic lockdown. We hypothesized that FEDUs and EDSUs would increase their ED use during the emergency declaration. Our secondary outcomes were to determine whether unique patient demographics and encounter characteristics differed between non-frequent, frequent, and super users of ED services before and during the stay-at-home order.

Study Design and Setting
The FLOWERS study was a secondary analysis of existing data for ED visits to a charitable Midwest healthcare system. This not-for-profit system has a network of 20 EDs (hospital-based, including a Level I and Level II trauma center and freestanding EDs) spanning 47 counties in Ohio Where Have All the FLOWERS Gone?
Levitin et al.
with a combined annual ED census of 492,650 visits by patients ≥18 years of age. The study was conducted with the approval of the hospital's institutional review board. The interval for the analysis included a designated 12-month period before and a 9-week interval during the mandatory COVID-19 stay-at-home order. Ohio's stay-at-home order was implemented on March 23, 2020. The order included ceasing operation of all non-essential business, prohibition of all public and private gatherings, limitation of travel, closure of schools, cancellation of elective medical procedures, and implementation of social-distancing measures. Therefore, March 23, 2020, was selected as the initial reference point to begin assessing the impact of this public health measure on peoples' willingness to seek ED care during the beginning of the mandatory lockdown. The cancellation of the order on May 29, 2020, was the end date in the study because it represented a transition point between lockdown and the resumption of business activities. The purpose of this study was to determine the impact of the mandatory COVID-19 pandemic stay-at-home order on the proportional makeup of ED visits by FEDUs and EDSUs. We used the total number of ED visits each registered person made over the prior 12 months (March 23, 2019-March 22, 2020) to categorize each patient into one of three user groups ( Figure 1). We then compared the proportional makeup  of each group during the prior 12 months to the 9-week emergency declaration period (March 23-May 29, 2020).
The study compared patient and admission characteristics for both periods. This information included ethnicity, race, ED disposition, insurance status, and arrival means. During the ED registration process, each patient's ethnicity was determined by inquiring whether they identified as being Hispanic or non-Hispanic. Patients were also asked about their race identification. Patients who self-identified as White or Caucasian were entered as the former, while the options Black or African American were entered as both. Patients who did not provide their race were given the following choices: White, African American or Black, American Indian or Alaska Native, Asian, Native Hawaiian or other Pacific Islander, or other designation (unknown, declined to specify, or two or more races). We collected race and ethnicity data to determine whether specific groups were impacted differently during the COVID-19 stay-at-home order.

Selection of Participants
Subjects for this study included 280,053 eligible individuals who registered to be evaluated in any of the 20 designated EDs during a 12-month period (March 23, 2019-March 22, 2020). These patients were identified by an electronic health record (EHR) query performed by a trained data analyst in the Quality and Patient Services (QPS) Department. The study excluded participants if they were <18 years of age, registered as John/Jane Does, were designated as a hospital transfer who never arrived at one of the study locations, left without being seen by a clinician, or were evaluated in an urgent care facility. The principal investigator and designated study staff reviewed a subset of records for inclusion/exclusion criteria to determine the accuracy of the QPS data query. The identified participants were subdivided into three groups based on their number of visits during the 12 months preceding the stay-at-home order: NFEDUs (<4 visits); FEDUs (4-9 visits); and SEDUs (≥10 visits). For each patient within the three groups, we compared their prior ED use to their ED visits during the Ohio stay-at-home declaration in response to COVID-19, a nine-week period from March 23-May 29, 2020.

Measurements and Analysis
Trained hospital data analysts collected patient and admission level data via a system-level query of the EHR (Epic Systems Corporation, Verona, WI) ( Table 1). The study team validated a subset of records using manual health records review to ensure data accuracy for each set of inclusion and exclusion criteria added to the query. Emergency department encounter characteristics of interest for this study included means of arrival, insurance status, and disposition, while patient-specific characteristics included gender, age, ethnicity, and race. We summarized all patient-and encounter-level data using means, percentages, and 95% confidence intervals. Missing data points were omitted from the calculation of percentages. For patients with repeat visits, we only reported data from a patient's first encounter during each period (before and during the stay-at-home order). We analyzed all data with R version 4.

Outcomes
We examined four outcomes in an analysis of ED use before and during the mandatory COVID-19 stay-athome lockdown: 1) proportion of NFEDUs (<3 annual ED visits); 2) proportion of FEDUs (4-9 annual ED visits); 3) proportion of EDSUs (≥10 annual ED visits); and 4) unique patient characteristics during the defined periods. These outcomes were chosen to determine whether the impact of the government-issued COVID-19 lockdown on ED visits were uniform across all patient demographics or whether there were specific subgroups whose ED use deviated from that of others. Figure 1 outlines the assignment of patients into three groups based on frequency of ED use. During this one-year period, 280,053 patients met the study criteria by registering for evaluation in one of 20 EDs within a single Midwestern healthcare system. Most patients (91.5%) were designated as NFEDU, while 7.8% and 0.7% were classified as FEDU and EDSU, respectively.

Patient Demographics and Encounter Characteristics Pre-lockdown
A summary and comparison of demographic data from each group for the one-year preceding the COVID-19 lockdown period (ie, pre-lockdown) is presented in Table 2. In general, patient demographics between the groups were similar. Most of the ED patients in each group were female (55-62%), White (approximately 70%), and of a similar age range, with over 92% self-identified as neither Hispanic nor Latino. Those of Black descent had a slightly higher representation among FEDU and EDSU patients (25.6% and 27%, respectively) than the NFEDU patients (20.2%). Asian Americans demonstrated a corresponding reduction in the two groups (FEDU, 1.3%; EDSU, 0.9%; NFEDU, 1.9%).
Government-sponsored programs (Medicaid and Medicare) provided health insurance coverage to 52% of NFEDUs, 74% of FEDUs, and 83% of EDSUs. Individually, Medicaid use was less prevalent in NFEDUs (26.5%) than FEDUs and EDSUs (45.2% and 52.6%, respectively), while the use of Medicare among all groups remained steady (25.6-30.7%). Private insurance was used by 40.5% of FEDUs, 18% of NFEDUs, and 9.8% of EDSUs. Compared to NFEDUs, FEDUs and EDSUs were more likely to be uninsured (2.8% vs 5.5% and 5.3%, respectively). In addition, EDSU patients were more likely to be hospitalized, leave against medical advice, leave without being seen after triage, and leave before final disposition. The EDSU patients arrived at the ED by ambulance more frequently than NFEDU and FEDU patients (26.8% vs 17.9% and 19.6%, respectively). The EDSU patients used public transportation to arrive at the ED more often than NFEDU and FEDU patients (3.4% vs 0.9% and 1.4%, respectively), and EDSU patients arrived by a personal vehicle less commonly than NFEDU and FEDU patients (65% vs 79.1% vs 76.1%, respectively).

Pre-Lockdown vs Lockdown
To determine whether the percentage of patients in each usage group changed during the mandatory lockdown, we compared the proportional makeup of each group between the two time periods. We included only known users who previously registered for ED care in the prior 12 months in this data analysis. Throughout the mandatory COVID-19 lockdown, the percentage of registered patients previously identified as FEDUs climbed from 7.8% (pre-lockdown) to 21.8% during lockdown (a 179% increase). The number of EDSUs grew by 571% (0.7% to 4.7%), while the number of NFEDUs dropped from 91.5% to 73.4% (a 19.8% decrease) during the lockdown (Table 3).
In comparing the proportions of ED visits for each group, we found that NFEDUs comprised 66.8% of all visits (a 6% decrease), FEDUs comprised 23.7% of all visits (a 3.9% increase), while EDSU encounters (9.5%) increased by 53% (  combined groups accounted for 26.5% of ED patients and 33.2% of ED encounters during the lockdown. We also compared demographics and patient-level encounter characteristics between the two periods (Tables 5 and 6). We saw a slight decrease in the proportion of male patients who registered for ED care during lockdown (41.4% vs 44%), with a corresponding increase in female registrants (58.6% vs 56%). During the lockdown, the percentage of patients who self-identified as White reduced from 70.4% to 67.7%, while those identifying as either African American or Black increased   from 20.8% to 25.4%. Age and ethnicity remained relatively unchanged before and during the lockdown. The proportion of individuals with private insurance dropped from 38.4% during pre-lockdown to 22% during lockdown (a decrease of 42.7%), while the percentage of uninsured ED visitors rose from 3.1% to 9% (an increase of 190.3%). The proportion of patients covered by Medicaid increased from 28.2% to 39.3% (an increase of 39.3%), while the proportion of Medicare users remained essentially unchanged. A greater proportion of patients arrived by ambulance during the lockdown than before the lockdown (22% vs 18%), while fewer were transported to the ED by private vehicle (73.2% vs 78.7%). Patient disposition varied little between the two periods except for a slight increase in the proportion of those who left against medical advice (1.4% vs 1.1%), those who were hospitalized (21.4% vs 19.4%), and those patients sent to labor and delivery (0.1% vs 0.04%) during the lockdown.
A total of 16,190 new patient encounters occurred during the lockdown. These ED visits were by individuals who had not previously registered for care during the prior year. As a result, these new patients were not included when calculating patient encounters and demographic data during the lockdown.

DISCUSSION
In 1955, the legendary songwriter and folk singer Pete Seeger wrote "Where Have All the Flowers Gone?"-an antiwar song about how wars can destroy an entire generation of young people. In the lyrics, young girls picked flowers to put on their boyfriends' graves, all of whom had died in battle. The song was translated into over 30 languages and helped define a generation. 18 Like war, pandemics also impact generations of people. The spread of SARS-CoV-2 resulted in over 255 million cases of COVID-19, and five million deaths worldwide, including 47.5 million cases in the US and over 768,000 deaths as of November 18, 2021. 19 The virus caused widespread economic hardships and exacerbated long-standing systemic health and social inequalities, placing individuals from racial and ethnic minorities at higher risk of getting sick and dying from COVID-19. 20,21 To curb the spread of disease, most governors throughout the US implemented mandatory stay-at-home orders. Healthcare systems followed suit by canceling elective procedures and limiting clinic and private practice hours. These actions, combined with the practices of social distancing, remote working, business, and school closures, and diminished vehicular usage, likely contributed to the nearly 40% curtailment in ED visits throughout the country. This reduction in ED patient volume during the declared national emergency appeared to impact all demographics and led to a uniform drop in routine, non-COVID-19-related medical emergencies (eg, myocardial infarctions, strokes, chronic obstructive pulmonary disease exacerbations, and critical patient admissions). [3][4][5][6]22 This dramatic change prompted our question: "Where have all the patients gone?" The FLOWERS study was a large, multicenter, singlestate, healthcare system retrospective investigation of the impact of the initial COVID-19 pandemic stay-at-home order on ED use by historically frequent users of emergency services. The objective was to determine whether the reduction in visits during this period was uniform for all patients, including those who historically are frequent users of ED services. We gathered data from 20 EDs (hospital-based and freestanding) spanning 47 counties, including 280,053 patients with 492,650 ED visits.
The FEDUs are often chronically ill individuals with several active comorbidities, socioeconomically disadvantaged, and high users of both ED and outpatient services. Any barrier to accessing routine medical care, such as the COVID-19 stayat-home order, should have increased ED visits from all patient groups, especially among frequent users of these services. However, we found that while patients who historically used ED services frequently did so more often during the lockdown, patients who were not frequent users tended to use ED services Other 6.6 (6.5-6.7) 4.9 (4.6-5.2) -25.8% CI, confidence interval; PI, Pacific Islander.  even less during the lockdown. The increased use among FEDUs and EDSUs was likely due to the reduced availability of other healthcare options.
Public health measures to reduce the spread of SARS-Co-V-2 likely impacted healthcare systems' ability to provide routine medical services such as disease screening, health

Levitin et al.
Where Have All the FLOWERS Gone? maintenance therapy, and mental health counseling. These community-based mitigation efforts, combined with deferred and delayed presentations of non-pandemic-related illnesses and pathologies, had negative implications worldwide. 23 According to the World Health Organization, 42% of countries had disruptions in cancer care, 49% had disruptions in diabetes care, and 31% for cardiovascular disease services during the early phase of the pandemic. These routine clinical services reductions likely contributed to excess deaths from treatable and preventable non-COVID-19-related health conditions and illnesses. [24][25][26][27] In addition, barriers to accessing routine healthcare services were likely the impetus for frequent users to seek out the ED in more significant numbers during the COVID-19 lockdown. Suppose these barriers and regular healthcare avoidance behaviors continued because of ongoing infection spread. In that case, patients could likely miss opportunities for acute medical interventions and necessary ongoing management of chronic conditions, vaccinations, and early screening for new medical problems that could worsen outcomes. 25 Delays or avoidance in seeking medical care might have also contributed to excess deaths during lockdown periods. A web-based survey conducted from June 24-30, 2020 estimated that 40.9% of US adults aged ≥18 years avoided care during the pandemic due to concerns over COVID-19. This forestalling included 12% who avoided urgent or emergency care and 31.5% who avoided routine care. Avoidance of urgent and emergent care was highest among unpaid adult caregivers, individuals with two or more underlying health conditions, persons with health insurance, Black and Hispanic patients, young adults, and persons with disabilities. Those falling under one or more of these categories also represent those who were at increased risk of developing severe COVID-19. 25,28 As a corollary, the reduction in healthcare utilization during the mandatory stay-at-home order may have had an unintended net positive effect on an individual's health. According to a physician survey conducted in 2017, an interpolated median of responses revealed that 20.6% of overall medical care might be unnecessary, including 22% of prescription medications, 24.9% of tests, and 11.1% of procedures. The top reasons cited for overtreatment included fear of malpractice (84.7%), patient pressure/request (59%), and difficulty assessing medical records (38.2%). 26 Also, individuals with higher incomes tend to undergo more expensive and extensive cancer screening exams to detect smaller abnormalities that lead to more follow-up testing and biopsies with little to no impact on mortality. 29 Additionally, as many as one-third of hospitalized patients may experience harm or an adverse event, often from preventable errors. In 2009, total excess costs in US healthcare exceeded $750 billion due to perceived unnecessary and inefficiently delivered services, excess administrative costs, too high prices, missed prevention opportunities, and fraud. 30 Data gathered from this study will be incorporated into future efforts to assess the impact of confounding factors to determine whether the lockdown disproportionately impacted the morbidity and mortality of NFEDUs and FEDUs. This ongoing research will also focus on community resources, family support systems, telemedicine, or other self-help strategies that either group may have used during the lockdown as an alternative to seeking emergent medical care. Such information may be useful in addressing these patients' needs in the future.

LIMITATIONS
There were several potential limitations to our study. There may have been an over-reporting of return visits in patients who were registered in the ED but left before completing evaluation only to return at a later time. This action could have increased their visit count by one, and if this patient visit was repeated, the accumulative effect might have incorrectly shifted them into one of the higher use groups. There is also a possibility that return visits may have been artificially elevated for patients whom the treating clinician requested that they return for a scheduled re-evaluation. These visits are typically not patientcentric decisions, as they are often the result of shared decisionmaking between the patient and the clinician. In addition, the exclusion of unidentified patients (ie, John/Jane Doe) from the study may have impacted the dataset if their number of ED admissions were significant. These individuals were each assigned a unique health record number that could be used to identify them upon return to the ED. The EHR system used by our health system is routinely updated to combine duplicate charts. If the data pull was redone now, there may be fewer John/Jane Doe in the dataset due to correction of errors by chart compilers. In our original dataset, there were 13 patients with John/Jane Doe status. Therefore, their inclusion in the study would not have impacted the results.
Another potential limitation involves the applicability of the study results to health systems in other states and locales. The government-issued, COVID-19 stay-at-home orders were not coordinated at the national level, which created the potential for implementation and impact variability. Although this variability was inherent to the process, the order's overall negative effect on patients' access to health services was generally uniform throughout the US. 17 For example, delaying elective procedures and organ transplants were a common patient surge management strategy deployed by most healthcare systems. 17 As a result of the stay-at-home order, dental offices closed and, in many states, nearly 80% of non-COVID-19 clinical trials were stopped or interrupted, including 400 clinical trials involving more than 200,000 cancer patients. 17 In addition, there was a marked reduction in preventive screening procedures (eg, colonoscopies, mammograms, and routine lab tests for the management of chronic disease) because of community mitigation measures during the early phase of the COVID-19 pandemic. 17 This study did not assess for confounding factors beyond the mandatory COVID-19 stay-at-home order that may have Where Have All the FLOWERS Gone?
Levitin et al.
impacted ED use, such as access to telemedicine, treatment advice offered online, and access to clinics that may have played a role in reducing the number of ED visits evaluated in the study. In addition, the data collected did not address other factors that may have influenced a patient's decision to seek repeated ED care, including limited access to health services in the evenings or on weekends and holidays, referral by a primary care physician or specialist, perceived quality of care, or insurance status. We hope to explore these confounding factors to assess their impact on patients' decisions to seek emergency care during a public health emergency in future work. We suspect that the study's large sample size mitigated the potential influence of these limitations.

CONCLUSION
It is incumbent upon clinicians and public health officials to better understand the impact on individuals' mental and physical health when society's most vulnerable are discouraged from using needed medical services during an infectious disease outbreak. An important part of this realization is addressing the implications of temporary disruptions in access to medical care during considerable periods of disease transmission. When barriers to accessing healthcare are implemented as part of a broader measure to reduce the spread of an infectious agent, individuals reliant on these services are more likely to seek out the ED for their medical needs. Future pandemic planning should consider this finding to ensure that vital healthcare resources are allocated appropriately. Conflicts of Interest: By the WestJEM article submission agreement, all authors are required to disclose all affiliations, funding sources and financial or management relationships that could be perceived as potential sources of bias. No author has professional or financial relationships with any companies that are relevant to this study. There are no conflicts of interest or sources of funding to declare.