The Annals November 2013 Journal Club issue marked one of the first collaborations with Academic Life in Emergency Medicine, a medical education blog, in an effort to promote a worldwide, transparent, online effort to perform critical appraisals of journal articles. The Global Emergency Medicine Journal Club was hosted on the blog for 1 week during November 18 to 24, 2013, with comments moderated on the blog and on Twitter. This summary article compiles the discussion and insights.

Study objective

Annals of Emergency Medicine collaborated with an educational Web site, Academic Life in Emergency Medicine (ALiEM), to host an online discussion session featuring the 2014 Journal of the American Medical Association publication on the Age-Adjusted D-Dimer Cutoff Levels to Rule Out Pulmonary Embolism (ADJUST-PE) trial by Righini et al. The objective is to describe a 14-day (August 25 to September 7, 2014) worldwide academic dialogue among clinicians in regard to 4 preselected questions about the age-adjusted D-dimer cutoff to detect pulmonary embolism.

Methods

Five online facilitators hosted the multimodal discussion on the ALiEM Web site, Twitter, and Google Hangout. Comments across the social media platforms were curated for this report, as framed by the 4 preselected questions, and engagement was tracked through various Web analytic tools.

Results

Blog and Twitter comments, as well as video expert commentary involving the ADJUST-PE trial, are summarized. The dialogue resulted in 1,169 page views from 391 cities in 52 countries on the ALiEM Web site, 502,485 Twitter impressions, and 159 views of the video interview with experts. A postdiscussion summary on the Journal Jam podcast resulted in 3,962 downloads in its first week of publication during September 16 to 23, 2014.

Conclusion

Common themes that arose in the multimodal discussions included the heterogeneity of practices, D-dimer assays, provider knowledge about these assays, and prevalence rates in different areas of the world. This educational approach using social media technologies demonstrates a free, asynchronous means to engage a worldwide audience in scholarly discourse.

Background

Free Open-Access Medical education (FOAM) use among residents continues to rise. However, it often lacks quality assurance processes and residents receive little guidance on quality assessment. The Academic Life in Emergency Medicine Approved Instructional Resources tool (AAT) was created for FOAM appraisal by and for expert educators and has demonstrated validity in this context. It has yet to be evaluated in other populations.

Objectives

We assessed the AAT's usability in a diverse population of practicing emergency medicine (EM) physicians, residents, and medical students; solicited feedback; and developed a revised tool.

Methods

As part of the Medical Education Translational Resources: Impact and Quality (METRIQ) study, we recruited medical students, EM residents, and EM attendings to evaluate five FOAM posts with the AAT and provide quantitative and qualitative feedback via an online survey. Two independent analysts performed a qualitative thematic analysis with discrepancies resolved through discussion and negotiated consensus. This analysis informed development of an initial revised AAT, which was then further refined after pilot testing among the author group. The final tool was reassessed for reliability.

Results

Of 330 recruited international participants, 309 completed all ratings. The Best Evidence in Emergency Medicine (BEEM) score was the component most frequently reported as difficult to use. Several themes emerged from the qualitative analysis: for ease of use-understandable, logically structured, concise, and aligned with educational value. Limitations include deviation from questionnaire best practices, validity concerns, and challenges assessing evidence-based medicine. Themes supporting its use include evaluative utility and usability. The author group pilot tested the initial revised AAT, revealing a total score average measure intraclass correlation coefficient (ICC) of moderate reliability (ICC = 0.68, 95% confidence interval [CI] = 0 to 0.962). The final AAT's average measure ICC was 0.88 (95% CI = 0.77 to 0.95).

Conclusions

We developed the final revised AAT from usability feedback. The new score has significantly increased usability, but will need to be reassessed for reliability in a broad population.