BACKGROUND: Prior data suggests the Mindfulness-Based Interventions: (MBI) Teaching Assessment Criteria (MBI:TAC) has good inter-rater reliability, but many raters knew teacher experience level. OBJECTIVE: We sought to further evaluate the MBI-TACs inter-rater reliability and obtain preliminary data on predictive validity. METHODS: We videorecorded 21 MBSR teachers from academic and community settings. We trained 19 experienced MBI teachers in using the MBI:TAC. MBSR teachers were rated by three assessors; teachers and their assessors did not know one another. To assess predictive validity, MBSR students in courses taught by 18 of the MBSR teachers were invited to complete PROMIS-29 measures before the MBSR course, at the end of the course (month 2), and month 4. RESULTS: Intraclass correlation coefficients (ICCs) representing a single rater ranged from 0.33 to 0.56 on the 6 MBI:TAC domains. Using an average of two raters, ICC estimates ranged from 0.48 to 0.71 and ICCs generalizing to an average of three raters ranged from 0.6 to 0.8. Among n = 152 participating MBSR students, we found improvements from baseline to 2 months and 4 months in PROMIS measures of Anxiety, Depression, Fatigue, Sleep, and Social Role function (range in improvement 2.3 to 6.3, P < 0.0001 for all comparisons except Social Role at 2 months, P = 0.007). Higher MBI:TAC ratings were associated with greater improvements in anxiety among MBSR students from baseline to 2 months, with a -0.31 lower participant anxiety score per 1 unit increase in MBI:TAC composite teaching rating (95% CI -0.58, -0.05, P = 0.019), but we did not find statistically significant relationships with improvements in other PROMIS-29 domains. CONCLUSIONS: ICCs indicated good reliability using an average of three ratings, but inter-rater reliability was only fair using a single rater. We found initial validation that higher MBI:TAC ratings predicted greater improvements in anxiety symptoms in MBSR participants.

Background

Mindfulness-based cognitive therapy (MBCT) is an effective group intervention for reducing rates of depression relapse. However, about one-third of graduates experience relapse within 1 year of completing the course.

Objective

The current study aimed to explore the need and strategies for additional support following the MBCT course.

Methods

We conducted 4 focus groups via videoconferencing, two with MBCT graduates (n = 9 in each group) and two with MBCT teachers (n = 9; n = 7). We explored participants' perceived need for and interest in MBCT programming beyond the core program and ways to optimize the long-term benefits of MBCT. We conducted thematic content analysis to identify patterns in transcribed focus group sessions. Through an iterative process, multiple researchers developed a codebook, independently coded the transcripts, and derived themes.

Results

Participants said the MBCT course is highly valued and was, for some, "life changing." Participants also described challenges with maintaining MBCT practices and sustaining benefits after the course despite using a range of approaches (ie, community and alumni-based meditation groups, mobile applications, taking the MBCT course a second time) to maintain mindfulness and meditative practice. One participant described finishing the MBCT course as feeling like "falling off a cliff." Both MBCT graduates and teachers were enthusiastic about the prospect of additional support following MBCT in the form of a maintenance program.

Conclusion

Some MBCT graduates experienced difficulty maintaining practice of the skills they learned in the course. This is not surprising given that maintained behavior change is challenging and difficulty sustaining mindfulness practice after a mindfulness-based intervention is not specific to MBCT. Participants shared that additional support following the MBCT program is desired. Therefore, creating an MBCT maintenance program may help MBCT graduates maintain practice and sustain benefits longer-term, thereby decreasing risk for depression relapse.

Background

The Mindfulness-Based Interventions: Teaching Assessment Criteria (MBI:TAC) is a widely used tool for assessing fidelity in mindfulness-based program (MBP) research and training. It also supports MBP teacher reflective and skill development. MBI:TAC assessors review MBP teaching and rate the teaching on 6 domains. The MBI:TAC yields individual domain and overall scores, using 6 levels of competence. Although the MBI:TAC is widely used in MBP research and training, research is at an early stage.

Objective

We developed and tested a method of training MBI:TAC assessors to use the tool reliably and examined interrater reliability of the tool.

Methods

A total of 31 international senior MBP teachers were recruited to join an online training to build their skills in using the MBI:TAC. The training systematically and iteratively built familiarity and skills in assessing the 6 MBI:TAC domains. Qualitative and quantitative data on trainee's experience of the training were gathered. Interrater reliability in using the tool was tested each week of the training. At the end of the training, interrater reliability was tested by asking trainees to individually assess videos that they had not previously seen. Their ratings were compared to benchmark assessments, which had been established via consensus agreement between 4 expert users of the MBI:TAC.

Results

The training was well received and appreciated, with some challenges experienced in applying the assessment methodology. Participants' ratings became progressively more in line with one another and the benchmark ratings during the training. At the end, interrater reliability was high (ranging from 0.67 to 1.0).

Conclusion

It is possible for senior MBP trainers, coming from different regions in the world, to align toward common understandings of the elements of MBP teaching competence and program integrity. An assessor training methodology was tested, and the learning from this project has led to refinements for future delivery.

Background

The Mindfulness-Based Interventions: Teaching Assessment Criteria (MBI:TAC) is an important tool for assessing teacher skill and aspects of the fidelity of mindfulness-based interventions, but prior research on and implementation of the MBI:TAC has used video recordings, which can be difficult to obtain, share for assessments, and which increase privacy concerns for participants. Audio-only recordings might be a useful alternative, but their reliability is unknown.

Objective

To assess evaluator perception of the rating process and inter-rater reliability of MBI:TAC ratings using audio-only recordings.

Methods

We prepared audio-only files from video recordings of 21 previously rated Mindfulness-Based Stress Reduction teachers. Each audio recording was rated by 3 trained MBI:TAC assessors drawn from a pool of 12 who had previously participated in rating the video recordings. Teachers were rated by evaluators who had not viewed the video recording and did not know the teacher. We then conducted semi-structured interviews with evaluators.

Results

On the 6 MBI:TAC domains, the intraclass correlation coefficients (ICCs) for audio recordings ranged from .53 to .69 using an average across 3 evaluators. Using a single rating resulted in lower ICCs (.27-.38). Bland-Altman plots showed audio ratings had little consistent bias compared to video recordings and agreed more closely for teachers with higher ratings. Qualitative analysis identified 3 themes: video recordings were particularly helpful when rating less skillful teachers, video recordings tended to provide a more complete picture for rating, and audio rating had some positive features.

Conclusions

Inter-rater reliability of the MBI:TAC using audio-only recordings was adequate for many research and clinical purposes, and reliability is improved when using an average across several evaluators. Ratings using audio-only recordings may be more challenging when rating less experienced teachers.