UC Berkeley

My dissertation research explores the factors that drive college STEM instructors’ adoption of equitable teaching practices, such as Adaptive Equity-Oriented Pedagogy (AEP). AEP is a framework for adjusting teaching to address equity barriers to learning based on student data collected through formative assessment, observations, and surveys (Phuong et al., 2017a). Through randomized controlled trials, I had previously found that compared to an active learning control course, instructors applying AEP improved average student achievement by over a letter grade and narrowed achievement gaps for all students (Phuong & Nguyen, 2019; Phuong et al., 2022).

In my dissertation research, 129 student-instructors (henceforth “instructors”) were randomly assigned to treatment and control pedagogy courses. While the control course taught instructors about AEP, the treatment course modeled AEP explicitly by using weekly instructor reflection data to continuously adjust course discussion and activities. Using multilevel regression modeling, I found that the treatment, on average, significantly improved instructors’ AEP competencies over time compared to the control (2.63 standard deviations, p<0.001), when controlling for instructors’ gender, URM status, years of teaching experience, years of tutoring experience, and number of students. Compared to the control condition, more instructors in the treatment condition used multiple data sources to adjust teaching and address equity barriers to advance student learning. This research demonstrates that modeling AEP principles explicitly (e.g., reviewing and addressing instructors’ beliefs and lenses based on written reflections) in educational development programs can improve instructors’ equitable teaching competencies.

Through the novel application of mixed-methods research and validated AEP measures, this study also offers a comparative analysis of a subset of instructors who exhibited low- versus high-growth on AEP competency. My research identifies the rationales instructors provided in their written statements for enacting particular teaching practices. These rationales were analyzed for evidence of a variety of factors driving instructors’ learning, such as their use of student data, conceptions of equity, emotions, and sharing the responsibility for learning. Compared to low-growth instructors, I found that high-growth instructors more frequently shared the responsibility for learning with their students, and used multiple data sources (e.g., formative assessment, observations, surveys) to understand equity barriers (e.g., imposter phenomenon, stereotype threat) and to adopt teaching practices that address these barriers.

My dissertation validates an Adaptive Equity-Oriented Pedagogical Competency (AEPC) assessment, a measure of college instructors’ effectiveness with inclusive teaching, that was used to track instructor growth in equitable teaching competencies in my study. The AEPC assessment includes reflection questions that ask instructors to provide evidence of demonstrating AEP competencies, such as clarifying learning outcomes; aligning formative assessments and activities with outcomes; understanding student equity barriers; and continuously adapting teaching to address these barriers. More specifically, the AEPC assessment applies a rubric to score self-reported data on instructors’ teaching philosophies, learning outcomes, written reflections on teaching and assessment practices, and instructor-provided evidence of inclusive pedagogies and student learning (e.g., student gain score visualizations, peer/student feedback on teaching). The AEPC assessment is thus built on a portfolio framework commonly used to evaluate college teaching (Seldin et al., 2010) but extended to incorporate well-defined criteria to measure equitable practices.

Based on partial credit Rasch modeling (Masters, 2010), the AEPC assessment has high reliability (0.94), has strong validity based on Wilson’s (2005) strands of validity framework, shows no gender assessment bias, has high inter-rater reliability, and correlates with researchers’ independent review of instructors’ teaching methods and pedagogical materials (Phuong et al., 2022). My research demonstrates that instructors can use the AEPC assessment to demonstrate competency and progress in applying equitable teaching practices. This assessment approach enables instructors to document evidence of inclusive teaching practices, which provides insight on how they might plan, teach, reflect, and alter their pedagogical practice to address students’ equity barriers. In addition, the AEPC assessment does not exhibit the known gender and racial bias found in course evaluations (Boring et al., 2016; MacNell et al., 2015; Phuong et al., 2022; Stark & Freishtat, 2014). My work contributes to conversations on how studies of college instructor professional development programs focused on equity can leverage randomized controlled trials, partial credit Rasch modeling, and multilevel modeling to reduce bias, mitigate the impact of confounding variables, create comparable groups, and support causal inference. This study provides recommendations for creating adaptive professional development programs that can advance equity in student outcomes.