Purpose

To evaluate research listed as "Submitted" on orthopaedic surgery residency applications for eventual publication rates and quality.

Significance

As the orthopaedic surgery residency selection process becomes increasingly competitive, the number of research publications listed on applications continually increases. However, the utility of using publications listed as "Submitted" in the applicant evaluation process remains unknown.

Methods

Demographic and publication data were retrospectively collected from 1303 applications to an orthopaedic surgery residency program. The PubMed database was used to verify "Submitted" publications for (1) publication fruition or (2) publication mismatch, defined as discordance between the listed journal of submission and the eventual journal of publication.

Results

A total of 594 applications (45.6%) listed ≥1 publication as "Submitted." Out of 1636 "Submitted" publications, 565 were unverifiable (32.5%). Of the 1071 verified publications, 362 (33.8%) experienced publication mismatch. Within this subgroup, a significant difference existed between the mean impact factors of the listed journal of submission and the eventual journal of publication (1.5 ± 2.7 versus 3.0 ± 2.5, P < 0.01). Demographic data were not predictive of having an unverified publication.

Conclusion

Publications listed as "Submitted" in orthopaedic surgery residency applications frequently remain unpublished or are published in less impactful journals than originally intended.

Background

Orthopaedic sports medicine fellowship positions are increasing in popularity, as evidenced by the increasing number of applicants to these programs. As positions have become more competitive, greater emphasis has been placed on an applicant's research experience. However, there has been a lack of research evaluating the accuracy of self-reported publications from fellowship applications.

Purpose

To evaluate the accuracy of self-reported research publications and the outcomes of studies submitted for publication by applicants to an Accreditation Council for Graduate Medical Education (ACGME)-accredited sports medicine fellowship in the United States (US).

Study design

Cross-sectional study.

Methods

Demographic and research publication data were retrospectively collected from 435 applications to an ACGME-accredited orthopaedic sports medicine fellowship program at a single high-volume academic institution from 2013 to 2017. All self-reported manuscript publications and studies in progress were analyzed with a minimum 2-year follow-up. "Submitted" publications were reviewed by searching the originally submitted journal and all publicly available sources. Publications were verified on PubMed, MEDLINE, and other open access journals. Journal impact factors were collected through use of InCites Journal Citation Reports.

Results

Only 5.7% (85/1504) of papers reported as "completed" were inaccurately self-reported, with 44 (51.8%) remaining unverified and 41 (48.2%) reporting discordant authorship, in which the published study listed a different author order than reported on the application. Further, 28.3% (197/696) of papers self-reported as "submitted" remained unpublished, 21.8% (152/696) were published in a different journal than originally reported, and 7.6% (53/696) were published with a different authorship order than reported. Among 95 applicants whose papers were published in different journals than originally reported, the mean impact factor of the final accepting journal was significantly lower than that of the journal of original submission (0.97 ± 0.13 vs 3.91 ± 0.79, respectively; 95% CI of the difference, 1.34-4.54; P < .01). Univariate analysis showed no significant relationships between variables of interest (age, sex, US Medical Licensing Examination Step 1 score, American Orthopaedic Association membership, medical school ranking, and advanced degree) and the presence of an inaccuracy.

Conclusion

There is a low rate of inaccurate self-reporting of "completed" publications on applications for orthopaedic sports medicine fellowships. The majority of papers listed as "submitted" on these applications were not published in the journals to which they were originally submitted.