Academic Primer Series: Key Papers About Teaching with Technology

Introduction Modern learners have immediate, unlimited access to a wide variety of online resources. To appeal to this current generation of learners, educators must embrace the use of technology. However, educators must balance newer, novel technologies with traditional methods to achieve the best learning outcomes. Therefore, we aimed to review several papers useful for faculty members wishing to incorporate technology into instructional design. Methods We identified a broad list of papers relevant to teaching and learning with technology within the online discussions of the Academic Life in Emergency Medicine (ALiEM) Faculty Incubator. This list was augmented with suggestions by a guest expert (BT) and an open call on Twitter (tagged with the #meded and #FOAMed hashtags) yielding 24 papers. We then conducted a modified three-round Delphi process within the authorship group, including junior and senior faculty members, to identify the most impactful papers. Results We pared the list of 24 papers to five that were most highly rated. Two were research papers and three were commentaries or editorials. The authorship group reviewed and summarized these papers with specific consideration to their value to junior educators and faculty developers. Conclusion This is a key reading list for junior faculty members and faculty developers interested in teaching with technology. The commentary contextualizes the importance of these papers for medical educators, to optimize use of technology in their teaching or incorporate into faculty development.


METHODS
In the fifth month of the ALiEM Faculty Incubator (July 2016), we discussed teaching and learning with technology and exchanged key literature that members felt were relevant to this topic. We monitored the proceedings of this community of junior and senior faculty members from July 1-31, 2016, and compiled the papers mentioned in these discussions.
To ensure that we had a broad compendium of articles, we augmented our collection with papers suggested by the month's guest expert (BT) and an open call for additional papers using Twitter with the hashtags #MedEd and #FOAMed. We subsequently conducted a three-round modified Delphi process to select papers relevant for faculty members on the month's topic, based on the opinion of the selected panel members. [12][13][14][15][16][17] Our selection panel included both novices (i.e., junior faculty members participating in the Faculty Incubator) and more experienced experts and educators (i.e., experienced clinician educators, all of whom have published >10 peer-reviewed publications, who serve as mentors and facilitators of the ALiEM Faculty Incubator), including the guest expert (BT, invited based on his past publications in the subject of teaching with technology). By mixing the opinions of the two groups, we sought to ensure that we selected papers of use to a spectrum of educators at different stages of their careers.

RESULTS
The ALiEM Faculty Incubator discussions and social media calls yielded a total of 24 articles. The three-round voting procedure allowed our team to generate a rank-order listing of all these papers in order of importance for faculty members. The citations and ratings are listed in Table 1.

DISCUSSION
The following is an annotated bibliography of the top papers, as determined by the modified Delphi process. The accompanying commentaries explain these papers' relevance to faculty members using these articles for personal or other faculty development.

Summary
New concepts and cultures within medical education make it difficult for educators to combine "technology-enhanced" education with traditional methods. This paper provides tips for reconciling the use of social media with more traditional education formats. The authors suggest that the principles for effectively using social media or podcasts in education are no different than other approaches: educators should provide clear learning objectives, assess learners, and evaluate teaching. Social media is one instructional strategy in education but cannot replace an entire curriculum.

Relevance to Junior Faculty Members
The paper provides valuable advice for junior faculty who are incorporating social media into their educational program. Junior faculty members should realize that newer, "technology-enhanced" methods of instruction are not automatically more effective. Many technologies are revisions of traditional methods and therefore must incorporate sound educational principles. For maximum efficacy, teaching with social media should use traditional educational objectives and program evaluation. Effective podcasting principles are similar to principles for an effective lecture. Curricula that incorporate social media should ensure that learners have study plans that include a wide range of topics, avoiding overemphasis of the popular  topics at the expense of common, less-exciting subjects. 19 Faculty members must help learners to consume FOAM in a manner similar to other medical literature. It may be valuable to assist learners in appraising resources or pre-selecting articles. In addition to serving as a method of receiving information, the authors highlight that social media may also have value as an outlet for reflection, feedback, and scholarly dissemination. Junior faculty should encourage learners to engage with social media and model examples of professionalism and confidentiality, as well as examples of quality academic scholarship. 20

Considerations for Faculty Developers
Faculty developers should consider expanding their mentorship of junior faculty members to include reflection on the faculty members' social media presence. Mentorship can be provided on professionalism and the effectiveness of educational programs. Faculty development programs may include instruction on how to expand faculty members' social media knowledge and skills. Social media may also be used directly as a medium for faculty development. Social media allows for the distribution of faculty development content as well as the formation of virtual communities of practice.

Summary
Disruptive innovations are radical paradigms that offer a simpler, more convenient, customizable, or cheaper solution to a problem and may provide solutions to many of the current medical education system issues. The authors propose that MOOCs and digital badges may disrupt the educational system. MOOCs are repositories of online content that can reach large audiences across institutional boundaries. They can share large amounts of content at a low cost and would be capable of expanding and standardizing the delivery of high-quality education materials. Digital badges are digitally encoded elements that students can earn to reflect mastery of skills or specific achievements, in place of traditional grades. The authors' vision for undergraduate and graduate medical education would use a central online collaborative learning environment and award transferable digital badges for competency-based assessments and advancement. The authors propose that such a system would facilitate interdisciplinary medical education, lifelong learning skills, and customization of learning outcomes. With less time devoted to didactic teaching, faculty could focus on higher level small-group discussions, observed assessments with formative feedback, and the verification of competency.

Relevance to Junior Faculty Members:
Junior faculty may fail to consider how their work fits into the bigger picture, preventing meaningful change in the field.  This editorial allows those junior faculty to see how work on disruptive innovations such as FOAM, MOOCs, digital badges, blended learning methodologies, and assessment methods could fit into the larger narrative of medical education evolution. While the discussion is idealistic, it may inspire junior faculty by introducing them to new, relatively uncharted fields within medical education.

Considerations for Faculty Developers
The paper examines the larger scale challenges in medical education. The authors predict that disruptive innovations and technology will play an important role in future changes in medical education, especially with respect to competencybased medical education. The paper provides a rationale for embracing change and potentially disruptive innovations. Operationalization of digital badges is supported by the growing focus within the medical education literature regarding Entrustable Professional Activities (EPAs) and Statements of Awarded Responsibility (STARs). 22 Faculty developers should be familiar with these concepts in order to inform and inspire junior educators.

Summary
The use of FOAM has rapidly increased in popularity among learners. Despite this, the academic community has been hesitant to fully endorse these educational materials because of uncertainty regarding their quality. This paper is the first collaborative effort by experts to develop consensus regarding quality of online medical education resources. The authors used a modified Delphi of expert emergency medicine and critical care bloggers and podcasters to determine the relative importance of each quality indicator for blogs and podcasts, from a previously defined list of 151 quality indicators pertaining to credibility, content, and design. 24 The authors invited expert participants (22 podcasters and 24 bloggers) to participate in the surveys, based on their position as lead editor(s) of one of the highest rated emergency medicine or critical care blogs/podcasts as determined by the Social Media Index. 25 The experts reached greater than 70% consensus for 85 quality indicators (5 for blogs only; 41 for podcasts; 39 of these for both), with greater than 90% consensus for 31 of these (5 for bloggers; 17 for podcasters; 9 for both).

Relevance to Junior Faculty Members
This resource provides a method to assess the quality of non-traditional educational resources, which are increasingly used by learners. While the list is not exhaustive, these quality indicators can assist learners and educators in evaluating online resource quality. Junior faculty educators may consider these indicators when deciding upon FOAM resources for their learners and/or curricula. Furthermore, these indicators may be useful to junior faculty members who wish to develop quality FOAM.

Considerations for Faculty Developers
As P&T committees begin considering FOAM in an educator's portfolio of scholarly activity, 11 faculty developers should be familiar with potential metrics for assessing FOAM quality. This paper is one of a few derivation and Delphi studies proposing quality indicators for FOAM resources. 20,[26][27][28] The paper may also foster a discussion about the lack of critical appraisal for other traditional secondary resources such as textbooks or narrative reviews.

Summary
Social media has rapidly emerged as a tool for disseminating medical innovations and education. Clinician educators engage in scholarship as part of their core mission. 29 However, traditional metrics for evaluating scholarly work are not readily applicable to social media-based scholarship. Furthermore, there are no publications identifying evaluation criteria or a formal definition for social media-based scholarship in medical education. While Thoma, et al. focused on quality metrics for blogs and podcasts, Sherbino et al. seeks to define criteria for them as scholarship. 20,23 Fifty-two health professions educators from 20 organizations in four countries reviewed various themes in medical education scholarship that had been previously identified by an expert working group. The group unanimously agreed on four key features of social media-based scholarship: 1) it must be original (i.e., cannot simply re-broadcast material created elsewhere); 2) it must advance the field by building on best practice, research, or theory; 3) it must be disseminated and archived; and 4) it must provide the education community with the opportunity to give transparent feedback that informs a wider discussion.

Relevance to Junior Faculty Members
Junior faculty members are under pressure to produce academic scholarship to further their careers and advance towards promotion and tenure. Traditionally, such scholarship is in the form of peer-reviewed research. At the same time, medical learners are increasingly using non-traditional sources for learning. This creates disconnect between the needs of the learner and educator. Junior faculty may be interested in becoming thought leaders in social media-based medical education but are held back by concerns about career advancement and academic recognition. By defining key qualities for this type of scholarship, junior faculty are provided with a framework to guide their scholarly efforts in these areas. As further consensus is achieved, social mediabased scholarship in health professions education will become a more defined part of academic advancement and contribution for clinician educators.

Considerations for Faculty Developers
Scholarship has and will continue to evolve. Social media-based resources have proven to be robust and durable ways to disseminate ideas that meet many of the traditional definitions of scholarship. Traditional, pre-publication peer review is a notable exception to this, however. Rigid adherence to such traditional paradigms is likely to limit faculty creativity and institutional flexibility. Faculty developers should consider the emerging role of social media as a form of scholarship if they wish to keep ahead of competing institutions and recruit tech-savvy educators and innovators. This paper provides a framework for the potential recognition of this work. This paper also is relevant to faculty who serve on their institutions' P&T committees. The metrics identified by this paper will help guide the acceptance and assessment of digital scholarship to be used in making promotion decisions.

Summary
This review article provides tips for engaging the millennial learner by optimizing the use of technology in resident education. The authors provide multiple suggestions for "flipping the classroom," including the importance of providing well-vetted, learner-responsible content (i.e., pre-class resources); maintaining active learner engagement through the use of team-based or problem-based learning; and using existing resources (e.g. MedEdPORTAL, Journal of Education and Teaching in Emergency Medicine, CORD Teaching Cases) when appropriate. When lectures are used, the authors suggest pre-reading, short lecture sessions, and active learning techniques (e.g. pause procedures, simulation, small-group discussions, audience response systems). Finally, the authors discuss optimizing the use of technology by having a central, cohesive repository for all residency information (e.g. learning management system); using technology to provide more effective feedback; and providing frequent learning opportunities through residency-run blogs and automated teaching pearls.

Relevance to Junior Faculty Members
Toohey et al. provide a number of valuable tips for the junior faculty member. With increasing emphasis on learner-centered education and the incorporation of online media into resident education, it is important to use these resources effectively. The authors emphasize active learning strategies supported by effective pre-reading when preparing didactics. Active learning can be achieved through numerous strategies including audience response systems, team-based learning, problem-based learning, and simulation. 9 When supervising residents on shift, the authors recommend rapid, real-time, formative feedback to avoid recall bias, enhance its impact on practice, and avoid surprising biannual summative assessments. Examples of technologicallyenhanced, real-time feedback software include Instant Eval, New Innovations, MedHub, and MyEvaluations.com. When supervising resuscitations or procedures, optical head-mounted displays (e.g., Google Glass™, GoPro™ cameras) or video recording can further reduce recall bias and allow for the supervisor to focus on the patient, while allowing for valuable feedback to be provided later.

Considerations for Faculty Developers
The article provides strategies for incorporating technology into a residency curriculum. Faculty development is critical to the success of any educational program. This article could act as pre-reading for faculty development seminars or be added to a reading list for junior faculty members. The article specifically focuses on effective teaching strategies, using technology to organize and manage a residency curriculum, and using technology for providing learners with feedback.

LIMITATIONS
First, the selection of articles for the modified Delphi process did not incorporate an exhaustive, systematic search of the literature. Rather, the knowledge of experts in the field was combined with crowd-sourced feedback from the online community to generate the list of articles that were evaluated. We believe that this process was able to identify some of the key papers on the topic of interest while keeping the list a manageable length for the participants in the modified Delphi process. However, it is possible that key articles could have been missed or excluded. Furthermore, the authors only considered journal articles, rather than including all sources on the subject of teaching with technology, such as blogs and podcasts.
Second, in a pure application of the Delphi technique all participants would be experts in faculty development. We declined this approach, opting instead to recognize the expertise of a range of faculty members. By incorporating junior faculty, we felt that we were better able to determine what is most relevant to them.
Third, the inclusion of authors within the field may have introduced bias in the selection of the articles. Being experts in Boysen-Osborn et al.
Academic Primer Series: Key Papers About Teaching with Technology the subject, the senior faculty were authors of many of the papers considered in the Delphi process. This limitation was balanced with the use of junior faculty who have not authored within the field and may have benefited article selection by accessing the expertise of these same authors. Finally, given that all of the participants in the modified Delphi process were involved in the Faculty Incubator in some way, it is likely that they were not a representative sample of academic emergency medicine clinicians.

CONCLUSION
This paper describes some key papers that may be useful to junior faculty members and faculty developers interested in teaching with technology. We believe that it will be helpful for clinician educators who seek to use technology effectively as the field of medical education continues to evolve at a rapid pace.