Center for Educational Development and Research

Background: Increasingly, anesthesiology has been embracing the importance of nontechnical skills as an important component of error prevention and management strategy (1). Various paradigms exist with considerable overlap, none likely superior to the next (3). The Anesthetists' Nontechnical Skills focusing on teamwork, task management, decision-making, and situation awareness (2). Crisis Resource Management includes dynamic decision-making, interpersonal behavior, and team management (3). The concept of cognitive errors, or faulty thought processes which lead to errors despite available knowledge to make the correct decision, address the behavioral psychology of decision making (4). We designed and evaluated an assessment tool to represent a synthesis of these concepts. Methods: Categories and behaviors of nontechnical and cognitive skill were taken from the literature for content validity and synthesized into a 5 point Likert-style evaluation tool containing an overall performance score and 24 discrete behavioral questions. Anesthesiology residents (PGY 2-4) were observed during management of simulated emergencies (n=38); performance was rated by two previously trained and calibrated experts. Performance ratings were given after debriefing to allow for thought process exploration, since not all of the skills are directly observable. Results: Instrument reliability was demonstrated by Cronbach's alpha of 0.812. Discussion: The nontechnical and cognitive skills (NCTS) tool reflects principles of various proprietary nontechnical skills curricula, and adds the concepts of cognitive errors to the assessment of decision making. It is crucial to debrief prior to completing the tool, since information about participant thought process is revealed during debriefing. A key feature of cognitive errors is that they are distinct from knowledge gaps, and thus the evaluator must know whether a knowledge gap is present in order to classify an observed behavior as a cognitive error. This assessment tool can be used to reliably evaluate anesthesiology residents' performance of these important skills that are ubiquitous across clinical encounters, regardless of the nature of the emergency.

In 2006, two urban universities started a new 6-week, residential program designed to provide a pipeline of qualified disadvantaged students to medical or dental school. Using both qualitative and quantitative techniques, this paper describes an iterative self-study process in which discrepancies in perceptions on program objectives held by students and faculty were detected. Analysis of interviews and a focus group with faculty resulted in eight program objectives. Faculty and students rank-ordered seven of these according to importance using pair-wise comparisons. Although they agreed that “skill building” was the most important program objective, students and faculty differed significantly on others. Some of these differences were attributed to inadequate communication about the program.

Self assessment accuracy is an essential precursor to the development of self-directed learning- a desired goal of medical education. The purpose of this study was to determine medical students’ long term accuracy of performance prediction, evaluated in the context of formative assessments designed to provide practice for the summative assessment. Students were asked to predict their performance on the final exam over six formative assessments and at the end of the final exam. Students’ confidence levels were predictive of near assessments, but were uncorrelated with final exam performance. Moreover, the ratings were stable over time. These findings are discussed in context of medical students’ approach to learning in the second year of medical school.

BACKGROUND: Medical schools across the nation have turned attention to strengthening their curricula in geriatric medicine to prepare students for an aging nation. Effects of enhanced curricula need to be evaluated for program monitoring and continuous improvement.

OBJECTIVE: To evaluate the effect of an enhanced geriatrics curriculum on medical students’ knowledge.

DESIGN: Longitudinal cohort studies with cross-sectional historical comparisons.

PARTICIPANTS: The Class of 2005 students (n = 150) and a random sample of the Class of 2001 students (n = 43) as a historical control with the control group received a lesser amount of geriatrics training.

MEASUREMENTS: A validated 18-item geriatrics knowledge test assessing medical students’ knowledge about aging (6 items) and clinical geriatric management (12 items) was administered to the Class of 2005 students at the baseline and end of years 1, 2, and 3, and to the Class of 2001 students at the end of year 4.

RESULTS: The Class of 2005 students demonstrated a significant increase in the number of items answered correctly (M = 35% to M = 75%, p < .001) and a significant decrease in the number of items answered “Don’t Know” (M = 44% to M = 5%, p < .001). The cohort also showed a better performance at the end of year 3 (M = 75%) than the fourth-year control group (M = 71%, p = .009).

CONCLUSIONS: The enhanced geriatrics curriculum demonstrated an effect on improving medical students’ geriatrics knowledge.