We propose a dyadic Item Response Theory (dIRT) model for measuring interactions of pairs of individuals when the responses to items represent the actions (or behaviors, perceptions, etc.) of each individual (actor) made within the context of a dyad formed with another individual (partner). Examples of its use include the assessment of collaborative problem solving or the evaluation of intra-team dynamics. The dIRT model generalizes both Item Response Theory models for measurement and the Social Relations Model for dyadic data. The responses of an actor when paired with a partner are modeled as a function of not only the actor's inclination to act and the partner's tendency to elicit that action, but also the unique relationship of the pair, represented by two directional, possibly correlated, interaction latent variables. Generalizations are discussed, such as accommodating triads or larger groups. Estimation is performed using Markov-chain Monte Carlo implemented in Stan, making it straightforward to extend the dIRT model in various ways. Specifically, we show how the basic dIRT model can be extended to accommodate latent regressions, multilevel settings with cluster-level random effects, as well as joint modeling of dyadic data and a distal outcome. A simulation study demonstrates that estimation performs well. We apply our proposed approach to speed-dating data and find new evidence of pairwise interactions between participants, describing a mutual attraction that is inadequately characterized by individual properties alone.

Models of protein energetics that neglect interactions between amino acids that are not adjacent in the native state, such as the Gō model, encode or underlie many influential ideas on protein folding. Implicit in this simplification is a crucial assumption that has never been critically evaluated in a broad context: Detailed mechanisms of protein folding are not biased by nonnative contacts, typically argued to be a consequence of sequence design and/or topology. Here we present, using computer simulations of a well-studied lattice heteropolymer model, the first systematic test of this oft-assumed correspondence over the statistically significant range of hundreds of thousands of amino acid sequences that fold to the same native structure. Contrary to previous conjectures, we find a multiplicity of folding mechanisms, suggesting that Gō-like models cannot be justified by considerations of topology alone. Instead, we find that the crucial factor in discriminating among topological pathways is the heterogeneity of native contact energies: The order in which native contacts accumulate is profoundly insensitive to omission of nonnative interactions, provided that native contact heterogeneity is retained. This robustness holds over a surprisingly wide range of folding rates for our designed sequences. Mirroring predictions based on the principle of minimum frustration, fast-folding sequences match their Gō-like counterparts in both topological mechanism and transit times. Less optimized sequences dwell much longer in the unfolded state and/or off-pathway intermediates than do Gō-like models. For dynamics that bridge unfolded and unfolded states, however, even slow folders exhibit topological mechanisms and transit times nearly identical with those of their Gō-like counterparts. Our results do not imply a direct correspondence between folding trajectories of Gō-like models and those of real proteins, but they do help to clarify key topological and energetic assumptions that are commonly used to justify such caricatures.

Introduction

Trust between supervisors and trainees mediates trainee participation and learning. A resident (postgraduate) trainee's understanding of their supervisor's trust can affect their perceptions of their patient care responsibilities, opportunities for learning, and overall growth as physicians. While the supervisor perspective of trust has been well studied, less is known about how resident trainees recognize supervisor trust and how it affects them.

Methods

In this qualitative study, 21 pediatric residents were interviewed at a single institution. Questions addressed their experiences during their first post-graduate year (PGY-1) on inpatient wards. Each interviewee was asked to describe three different patient care scenarios in which they perceived optimal, under-, and over-trust from their resident supervisor. Data were analyzed using thematic analysis.

Results

Residents recognized and interpreted their supervisor's trust through four factors: supervisor, task, relationship, and context. Optimal trust was associated with supervision balancing supervisor availability and resident independence, tasks affording participation in decision-making, trusting relationships with supervisors, and a workplace fostering appropriate autonomy and team inclusivity. The effects of supervisor trust on residents fell into three themes: learning experiences, attitudes and self-confidence, and identities and roles. Optimal trust supported learning via tailored guidance, confidence and lessened vulnerability, and a sense of patient ownership and team belonging.

Discussion

Understanding how trainees recognize supervisor trust can enhance interventions for improving the dialogue of trust between supervisors and trainees. It is important for supervisors to be cognizant of their trainees' interpretations of trust because it affects how trainees understand their patient care roles, perceive autonomy, and approach learning.

The entrustment framework redirects assessment from considering only trainees competence to decision-making about their readiness to perform clinical tasks independently. Since trainees and supervisors both contribute to entrustment decisions, we examined the cognitive and affective factors that underly their negotiation of trust, and whether trainee demographic characteristics may bias them. Using a document analysis approach, we adapted large language models (LLMs) to examine feedback dialogs (N = 24,187, each with an associated entrustment rating) between medical student trainees and their clinical supervisors. We compared how trainees and supervisors differentially documented feedback dialogs about similar tasks by identifying qualitative themes and quantitatively assessing their correlation with entrustment ratings. Supervisors themes predominantly reflected skills related to patient presentations, while trainees themes were broader-including clinical performance and personal qualities. To examine affect, we trained an LLM to measure feedback sentiment. On average, trainees used more negative language (5.3% lower probability of positive sentiment, p < 0.05) compared to supervisors, while documenting higher entrustment ratings (+ 0.08 on a 1-4 scale, p < 0.05). We also found biases tied to demographic characteristics: trainees documentation reflected more positive sentiment in the case of male trainees (+ 1.3%, p < 0.05) and of trainees underrepresented in medicine (UIM) (+ 1.3%, p < 0.05). Entrustment ratings did not appear to reflect these biases, neither when documented by trainee nor supervisor. As such, bias appeared to influence the emotive language trainees used to document entrustment more than the degree of entrustment they experienced. Mitigating these biases is nonetheless important because they may affect trainees assimilation into their roles and formation of trusting relationships.

Context

Clinical supervisors make judgements about how much to trust learners with critical activities in patient care. Such decisions mediate trainees' opportunities for learning and competency development and thus are a critical component of education. As educators apply entrustment frameworks to assessment, it is important to determine how narrative feedback reflecting entrustment may also address learners' educational needs.

Methods

In this study, we used artificial intelligence (AI) and natural language processing (NLP) to identify characteristics of feedback tied to supervisors' entrustment decisions during direct observation encounters of clerkship medical students (3328 unique observations). Supervisors conducted observations of students and collaborated with them to complete an entrustment-based assessment in which they documented narrative feedback and assigned an entrustment rating. We trained a deep neural network (DNN) to predict entrustment levels from the narrative data and developed an explainable AI protocol to uncover the latent thematic features the DNN used to make its prediction.

Results

We found that entrustment levels were associated with level of detail (specific steps for performing clinical tasks), feedback type (constructive versus reinforcing) and task type (procedural versus cognitive). In justifying both high and low levels of entrustment, supervisors detailed concrete steps that trainees performed (or did not yet perform) competently.

Conclusions

Framing our results in the factors previously identified as influencing entrustment, we find a focus on performance details related to trainees' clinical competency as opposed to nonspecific feedback on trainee qualities. The entrustment framework reflected in feedback appeared to guide specific goal-setting, combined with details necessary to reach those goals. Our NLP methodology can also serve as a starting point for future work on entrustment and feedback as similar assessment datasets accumulate.