UC Davis

Objective

To compare how coder ratings of standardized patient (SP) visit recordings and SP ratings of the visits detect primary care physician (PCP) training in self-efficacy enhancing interviewing techniques (SEE IT).

Methods

Analyses of data from 50 PCPs who participated in a randomized controlled trial of SEE IT training, which led to increased SEE IT use during three SP visits 1-3 months post-intervention. Untrained SPs rated SEE IT use post-visit. Subsequently, three trained coders generated a consensus SEE IT rating from visit audio recordings. SPs and coders were blinded to provider study arm, and coders to SP ratings.

Results

SP and coder ratings were correlated (r=0.62). In detecting the intervention effect, the areas under the receiver operating characteristic curve were 0.80 (95% CI 0.74-0.87) and 0.76 (95% CI 0.69-0.84) for consensus coder and SP ratings, respectively (difference 0.04, 95% CI -0.04-0.11; z=1.04, p=0.30).

Conclusion

SP ratings were not significantly different from coder ratings of SP visit recordings in detecting PCP SEE IT training.

Practice implications

If similar findings are observed in larger studies, it would suggest a greater role for SP ratings in detecting provider interviewing skills training, given the relative simplicity, low cost, and non-intrusiveness of the approach.