INCREASING OFF-SERVICE RESIDENT PRODUCTIVITY WHILE ON THEIR EMERGENCY DEPARTMENT ROTATION USING SHIFT CARDS

—Background: Differences in productivity between off-service residents rotating in the emergency department (ED) and their emergency medicine (EM) resident counterparts have never been directly quantified. Objectives: We sought to quantify the difference between off-service residents rotating in the ED and their EM resident counterparts. We also sought to find whether shift cards could be used to increase the productivity of off-service res- idents rotating in the ED. Methods: This is a prospective cohort study conducted at an urban, tertiary, Level I trauma center. We implemented the use of shift cards for off-service residents during their EM rotation. Completion of the shift card involved recording patients seen and their dispositions, procedures done, and documenting a learned bedside teaching point from their shift that day. Productivity was measured in terms of patients seen per hour (PPH) and relative value units per hour (RVU/h). Results: Off-service res- idents showed a productivity of 0.529 PPH (95% confidence interval [CI] 0.493–0.566) and 1.40 RVU/h (95% CI 1.28–1.53) prior to implementation of shift cards. With the introduction of shift cards, productivity increased to 0.623 PPH (95% CI 0.584–0.663, p = 0.001) and 1.77 RVU/h (95% CI 1.64–1.91, p = 0.001). In comparison, first year EM resident productivity was 0.970 PPH (95% CI 0.918–1.02) and 3.01 RVU/h (95% CI 2.83–3.19). Conclusions: Shift cards can be used to foster motivation for off-service residents rotating in the ED, and is a simple and cost- effective method to improve system-based practices and utilization of resources.


INTRODUCTION
Emergency medicine (EM) has traditionally been a rotation where off-service residents gain important clinical experience through month-long rotations (1). However, in the face of declining Medicare reimbursement, graduate medical education funding is struggling to provide for an appropriate physician workforce without bankrupting teaching hospitals (2). In fact, many EM programs have had difficulty with funding off-service rotations and many have had decreased numbers of off-service res-idents in their Emergency Departments (EDs) (3). At the same time, experts predict that EDs will see a flood of new patients when the Medicaid expansion provisions go into effect, especially in communities with large uninsured populations (4). It is imperative that EM programs make efforts to ensure that off-service residents are properly utilized and have an adequate amount of experience for their training. Still, differences in productivity be-tween off-service residents and EM residents within the same institution have not previously been quantified, and efforts to increase off-service resident productivity have not been investigated.
We intend to use the shift cards to increase off-service resident productivity. Motivating offservice residents rotating in the ED through the shift cards can potentially be a simple and costeffective method to improve system-based practices and utilization of resources. Additionally, off-service resident education may be enhanced through the increased number of patient experiences as a result of increased work productivity. We seek to determine whether shift cards can be used to increase productivity in off-service residents rotating in the ED.

Study Design
This was a prospective cohort study conducted at an urban, tertiary, Level I trauma center with an annual census of 38,000 ED visits in 2011-2012. In our ED, off-service residents typically rotate in the ED for a month as part of their training. The study was conducted from January 2011 through December 2012. This study was approved by the University of California, Irvine Institutional Re-view Board on December 12, 2013.

Study Setting and Population
All rotating off-service residents rotating in the ED were included in this study. Off-service resident specialties included family medicine, anesthesiology, obstetrics and gynecology, otolaryngology, and psychiatry. All off-service residents were postgraduate year-1 (PGY-1) residents, with the exception of family medicine residents, who were all postgraduate year-2 (PGY-2) residents. All PGY-1 EM residents were also monitored, but did not receive the intervention.
We implemented the use of shift cards for off-service residents rotating in the ED from January 2012 through December 2012. Off-service residents were required to complete shift cards after each shift. Completion of the shift card involved recording patients that were seen and their dispositions, procedures done, and documenting a learned bedside teaching point from their shift that day. Shift cards were double-sided, making more patient slots than a resident could fill in a given shift. The total number of available lines to document patients seen was 18 (Figure 1). At the end of the shift, a senior EM resident or EM faculty member signed the shift card and provided feedback. Shift cards were then deposited in a locked collection box at the end of each shift and were collected on a weekly basis. Off-service residents were blinded to the study and were not encouraged by any means to in-crease their productivity. Residents were only told that they must complete the shift card after each shift to receive credit for their EM rotation.
We designed and implemented an online monitoring program that integrated our electronic health records with our electronic billing database. We used the electronic health records data to obtain the names of the resident physician associated with each patient. This information was then matched with the procedure billing data obtained from our electronic billing database, and the resident' s schedules. The supervising attending physician codes a patient billing form for each visit, including the International Classification of Diseases, 9 th Revision and Current Procedural Terminology codes. Relative value units (RVUs) and patients per hour (PPH) are calculated based on these forms, through the coding process in the billing department. Using Tableau Software (Seattle, WA), data were presented through an easily accessible online dashboard that could be downloaded as a spreadsheet. Thus, through this dashboard, we were able to monitor resident productivity on a per-visit basis. Resident clinical productivity was measured by calculating the average number of PPH and their total RVUs per hour (RVU/h). RVU/h encompasses not only volume of patients seen, but also the time, effort, skill, and resources required for each patient visit. Our primary outcome was to measure the change in PPH and RVU/h as compared between pre-and postintervention residents. We also compared PPH and RVU/h of off-service residents to that of PGY-1 EM residents.
Data were analyzed with SPSS Statistics 22.0 (IBM, Armonk, NY). We used one-way analysis of variance to compare pre-and postintervention data of off-service res-idents and their counterpart EM residents.

RESULTS
All off-service residents that rotated in the ED were included in the study. A total of 29 offservice residents received the intervention and were compared to 33 off-service residents that rotated in the ED prior to the intervention. A total of 12 PGY-1 EM residents were also monitored for additional comparison, but did not receive any intervention.
When correcting for ED patient volume, off-service residents showed a productivity of 0.529 PPH (95% confidence interval [CI] 0.493-0.566) and 1.40 RVU/h (95% CI 1.28-1.53) prior to implementation of shift cards. Productivity increased to 0.623 PPH (95% CI 0.584-0.663, p < 0.001) and 1.77 RVU/h (95% CI 1.64-1.91, p = 0.001) after the implementation of shift cards (Figures 2 and 3). There was no statistically significant difference in PGY-1 EM resident productivity between pre-and postintervention years as measured in RVU/h (p = 0.417). When performing a subgroup analysis by each specialty, the statistically significant increase in Figure 1. Shift card given to off-service residents. productivity was observed in the PPH outcome within Family Medicine (Table 1). Within offservice residents, rotation month and specialty did not have a statistically significant effect on PPH or RVU/h. In comparison, first-year EM resident productivity was 0.970 PPH (95% CI 0.918-1.02) and 3.01 RVU/h (95% CI 2.83-3.19). In comparison to off-service residents in the post-intervention group, first-year EM residents were still significantly more productive in terms of PPH (p < 0.001) and RVU/h (p < 0.001). We also found that increased hospital patient volume was positively correlated with the productivity of EM PGY-2 and EM PGY-3 residents, but not of off-service residents or EM PGY-1 residents ( Table 2).

DISCUSSION
Off-service residents from several specialties rotate through the ED as part of their training. This helps serve both as increasing the work force in the ED and providing these off-service residents with perspective into the workings of the ED, from where they usually tend to admit their patients.
This study was aimed to determine if there are differences in off-service resident productivity and whether implementing the use of shift cards could help motivate off-service residents to be more productive during their ED rotation. Based on our results, there is an increase in productivity post intervention in the off-service resident group. Clinically, this translates to approximately one more patient seen per shift and 4.44 RVUs. Each additional patient seen by a resident creates more procedural opportunities and experience gained during each shift. Based on the Centers for Medicare & Medicaid Services, one RVU has a reimbursement rate of $34.04, which would equate to a $408.48 increase over an entire shift (5). The EM PGY-1 residents continue to be more productive in both the number of patients seen and the acuity of patients treated. This could possibly be explained by more experience in the ED, as well as the familiarity of the environment. This is in contrast to a previous study, which found that off-service residents work up patients at the same rate as EM residents. However, in this study, EM residents and offservice residents were not from the same institution (6). Further, the increase in off-service resident productivity did not affect EM intern productivity. No intervention was implemented for the EM PGY-1 residents during this time. It is important to consider that the results could be different if the intervention was also implemented for the EM PGY-1 residents such that there would also be an increase in their PPH and RVU/h. Interestingly, we also found that off-service resident productivity, measured in RVUs/h, was negatively correlated with ED patient volume; however, EM PGY-2 and PGY-3 resident productivity were positively correlated with ED patient volume. Hence, when the ED saw greater patient volumes, PGY-2 and PGY-3 residents showed an overall increase in RVUs/h. This was not seen in PGY-1 or off-service residents. Similarly, Jeanmonod et al. found a positive correlation between EM residents and ED patient volume (7). This may suggest that the PGY-2 and PGY-3 EM residents are more adaptable to increased patient volumes and subsequently take a larger majority of the workload as the ED becomes more crowded or busy.
Furthermore, off-service resident specialty did not have a statistically significant effect on RVU/h or PPH in off-service residents. This suggests that off-service res-idents work at approximately the same level of productivity regardless of specialty. Stone et al. also found that specialty differences in the average number of patients seen per off-service resident were not statistically significant (6).

Limitations
We acknowledge that when analyzing the effect of the intervention on individual specialties, the study's sample size was relatively small. Further, we did not have any off-service residents from the Psychiatry Department in the postintervention group because their training no longer required them to rotate in the ED at our institution. There was also a slightly different distribution of off-service resident specialties that rotated in the ED, but again, specialty did not have a significant effect on resident productivity. General skill level and preparedness coming into intern year between the 2 years may have also impacted the results. However, no other changes in variables in comparing the two groups were made.
Additionally, RVU data relied on accurate coding of charts by the billing department and consistent entry of residents as caregivers. Some experiences may not have been recorded by the resident and therefore would not have contributed to their RVU. Furthermore, it is possible that off-service residents are not as thorough in their documentation as ED residents, and this might also influence the RVU for that particular visit.
Finally, we realize that there is a nonuniform distribution of patient flow within the ED. Certain days and time of day may have increased or decreased patient flow, which could affect resident productivity. But these effects would likely be dissipated over the entire study period.

CONCLUSIONS
Overall, the use of shift cards is a tool that can be used to foster motivation for off-service residents rotating in the ED, and is a simple and cost-effective method to improve system-based practices and utilization of resources. Ac-cording to the Accreditation Council for Graduate Medical Education, residency programs should seek to produce physicians who are capable of practicing medicine with a balance of compassion, excellence, and efficiency. Additionally, the Next Accreditation System aims to prepare physicians for practice in the 21 st century. This includes enhancing the competence of future physicians in areas that are relevant to a wellperforming, efficient, and cost-effective health care system (8). Our study highlights a simple method to increase resident efficiency so they may be more prepared to practice as independent physicians. Furthermore, increased productivity translates into more experiences (e.g., more patients seen) and may help ensure that residents get an adequate breadth of training. For example, more productive residents had a higher PPH, which equates to more experience. We do acknowledge that increased number of patient encounters does not necessarily ensure complete competency. However, it has been long documented in surgical residencies that more experiences (i.e., performing more cases) is the primary method in which technical competency is developed (9,10). This can be extended to other fields of medicine, including EM, in which residents may become more competent through increased exposure. Considerations for future research should include a larger sample size, more specialties, and possibly multi-center studies with implementation of shift cards for off-service residents.