Emergency Medicine Program Director Perceptions of the Resident Selection Process Following the Transition to a Pass/Fail USMLE Step 1
Abstract
Learning Objectives: To assess Emergency Medicine residency program directors’ perceptions of the impact of Step 1 P/F reporting on other selection criteria.
Background: Beginning in 2022, the NBME will transition score reporting of USMLE Step 1 from a 3-digit score to Pass/Fail (P/F). Historically, Step 1 has been weighed heavily by program directors (PDs) as an important metric in assessing competitiveness for residency. Our study examined whether EM program directors would place increasing value on Step 2 CK scores following the transition to a P/F Step 1.
Objective: To assess PD perceptions of the impact of Step 1 P/F reporting on other selection criteria.Methods: A survey consisting of ranking questions was sent to PDs of all 282 EM programs in the US. These questions focused on assessing current resident selection practices in comparison to expected selection criteria changes following transition to P/F Step 1. Sixteen attributes were ranked by importance by PDs based on their own applicant selection process. PDs were also asked a series of questions to determine their confidence in Step 1 and Step 2 CK scores in predicting an applicant’s clinical skills and ability to succeed.
Results: The survey was completed by 43 (15.24%) PDs. 52.6% reported that both Step 1 and Step 2 CK adequately predicted a resident’s ability to pass EM board examinations (52.6% and 73.7% respectively). When asked if Step 1 and Step 2 CK are accurate predictors of a resident’s ability to perform well clinically, only 10.5% of program directors answered yes to Step 1, compared to 31.6% for Step 2 CK. PDs ranked 14 of the 16 attributes higher following the transition to P/F Step 1 (p<0.001 per attribute). Step 2 CK score importance was ranked lower following transition to P/F Step 1. The first quartile of attributes pre- and post-transition did not change and are as follows: letters of recommendation [1st], away rotation [2nd], clerkship grades [3rd] and Step 2 CK score [4th].
Conclusion: Despite no longer having a 3-digit Step 1 score, PDs may not necessarily place greater emphasis on Step 2 CK scores following the transition.