Journal of Diversity and Equity in Educational Development

Quantitative Research

As one strand of the faculty development literature, quantitative studies have primarily focused on measuring change through standardized instruments administered at discrete points in time. Several studies focus on quantitative metrics to measure statistical growth in knowledge, self-efficacy, and pedagogical implementation. O’Connor et al. (2019) examined the impact of their three-day DEI institute using a quantitative pre- and post-institute survey design with an adapted DEI-related teaching self-efficacy measure, demonstrating statistically significant increases in nursing faculty’s confidence in engaging, supporting, and evaluating students with DEI content, and designing curricula with DEI content. Harrison-Bernard et al. (2020) also employed pre- and post-workshop surveys to analyze the impact of a three-hour workshop and found significant gains in participants’ ability to identify bias and use DEI terminology. Extending this outcome-focused approach, Campbell-Whatley et al. (2016) surveyed the faculty participants and non-participants of their five-day Diversity Institute to compare their use and confidence in diversity content in their teaching. Additionally, they surveyed the students of each group to compare their observations of diversity-related content in their classrooms. A statistically significant group difference was observed between the faculty groups in their use of diversity-related content. Also, the mean difference between the student groups in their observation of diversity content was statistically significant. Collectively, these studies demonstrate the value of survey-based designs for documenting short-term changes in attitudes, confidence, and teaching practice following DEI-focused PD.

Qualitative Research

In contrast to survey-driven approaches, qualitative studies capture the depth of participant experiences and the nuances of cultural shifts within institutions. Magalhães and Hane (2020) conducted semi-structured interviews and found that a multi-year learning community helped STEM faculty maintain or increase their awareness of classroom inclusion issues and implement inclusive teaching strategies. Booker et al. (2016) also conducted semi-structured focus group interviews with both faculty who participated in a week-long summer program and students taught by those faculty participants, demonstrating the program’s effectiveness in both improving faculty attitudes and in making curricular changes related to DEI topics. Students also observed a sense of community, personal growth, and improved conflict-resolution skills in the classes taught by the faculty. Kim et al. (2023) ran a series of PD programs and collected responses from their end-of-semester survey, which showed that diversity-infused professional development successfully helps faculty rethink their syllabi and teaching approaches. The PD programs also supported the promotion of Universal Design for Learning principles in course design. Finally, Hudson (2020) provided a unique ethnographic account through participant observation, chronicling her own transformative journey alongside interviews and surveys to assess the impact of a five-month program on teaching strategies. Together, these qualitative studies highlight the richness of faculty perspectives and the potential for PD to influence practice, while often relying on retrospective or single-time-point accounts.

Mixed-Methods

Seeking to bridge breadth and depth, a third group of studies combines quantitative and qualitative approaches. The studies utilize mixed-methods or descriptive reports to provide a holistic view of program effectiveness and departmental change. O’Leary et al. (2020) implemented a mixed-methods study using pre-, post-, and follow-up surveys combined with informal group feedback six months later. They demonstrated that the STEM faculty increased their knowledge of social identities and the barriers to learning in classrooms, shifted away from a fixed mindset about students’ ability, and modified their teaching approach. Ceo-DiFrancesco et al. (2019) analyzed participants’ feedback, including quantitative (Likert-scale responses) and qualitative (open-ended survey questions), at the conclusion of the 14-week Diversity and Inclusion Teaching Academy and learned that the faculty sought opportunities for critical dialogue and reflection and that the facilitators should model sound pedagogical practices. While these designs offer a more holistic picture of impact, they still tend to emphasize aggregated outcomes rather than participants’ ongoing interpretation and adaptation of their learning over time.

Taken together, this body of literature demonstrates that EDI-focused PD can influence instructors’ beliefs, confidence, and teaching practices. At the same time, much of this work relies on data collected immediately after PD experiences or aggregates participant responses in ways that obscure how faculty make meaning of, adapt to, and extend their learning over time. Notably, fewer studies examine unanticipated outcomes or the extended impact of EDI-focused PD efforts beyond the originally targeted course. This gap informs our choice of collaborative autoethnography to examine the longitudinal and holistic impacts that are difficult to surface through other approaches.

Reflexivity Statement

CAE recognizes researchers’ positionalities and roles as integral to interpretation and meaning-making. Across the author team we hold varied social identities and institutional positions that have shaped our engagement with EDI work and our interpretation of participants’ accounts. The first author identifies as a foreign-born Asian, cisgender, middle-class woman, and holds the institutional role of educational developer; as the cohort facilitator, she brought an insider perspective on the program’s goals, structure, and intended EDI-related outcomes. The second author identifies as a white American, cisgender female, and holds the institutional roles of faculty and endowed chair in the scholarship of teaching and learning. As a non-participant, she provided methodological and writing support and contributed analytic distance from the cohort experience. The third author identifies as a Vietnamese American, middle-class wife and mother, and holds the institutional role of faculty member; as a cohort participant, she brought a perspective grounded in lived experience implementing EDI-centered teaching practices. We acknowledge these identities and roles shaped what we noticed and how we represented impact. To account for this, we used iterative reflection, dialogic analysis, and collective sensemaking, with the second author intentionally posing disconfirming questions and pressing for clear links among claims, excerpts, and EDI-focused interpretations. Although our small sample limits generalizability, transparency about author roles and our collaborative analytic process strengthens the trustworthiness of our interpretations of the impacts of participation in an EDI-centered course redesign program.

Participants

In 2021, all 27 participants in the Reinvent Your Course for EDIA cohort were invited to be part of this study. Eight cohort members represented fine arts disciplines (arts, music, theatre, and dance), eight were from humanities disciplines (history, psychology, and communication), five were from business disciplines (marketing, management, and finance), and six were from various other disciplines. Ultimately, four individuals enrolled. Characteristics of each participant, including the discipline represented and the type of course redesigned, are provided in Table 2 below. Random names have been assigned to each participant to ensure their anonymity.

Method

This study was designed to better understand the lived experiences of participants who completed the Reinvent Your Course for Equity, Diversity, Inclusion, and Access cohort. To achieve this, CAE was used to obtain reflective accounts of participants’ experiences teaching their redesigned courses. CAE is a qualitative research method that allows scholars to work together to collect autobiographical data, then interpret those data collectively to gain a meaningful understanding of sociocultural phenomena reflected in their shared experience (Chang et al., 2013, p. 24). CAE centers each participant as both a subject and a scholar who can provide invaluable information about context, experiences, and perspectives that are critical to understanding a particular topic. In this case, to understand the immediate and delayed impacts of engaging in course redesign, participants were allowed to identify and discuss their experiences as instructors in a manner that synthesized and consolidated understandings in a systematic manner.

Data Collection

Data collection began after each participant completed their course design experience in the summer of 2021 and then taught their redesigned course in the following fall term. Each participant was asked to complete individual, written reflections about teaching their redesigned course every month during the fall term. Participants were asked to respond to the same questions for each reflection except for the first reflection at the beginning of the semester.

1. As the semester started, what was your experience like getting your reinvented course up and running? (asked only for Reflection period 1)

2. What have you done differently in your course that you had planned to change as a result of your experience reinventing your course?

3. What did you do differently in your course that you hadn’t planned to do differently as a result of your experience reinventing your course?

4. What reactions did you and/or your students have to the changes you integrated into your teaching/learning context?

These reflections were submitted to the first author of this paper and were archived for later group analysis.

During the first month of the spring term, a focus group was convened to gather data from the study participants as to their experiences during and after their redesign cohort experience. The intention was that the focus group would serve as a debrief of the course redesign experience and the use of elements of the redesign cohort in teaching the course. The debriefing focus group was conducted synchronously via Zoom, allowing all participants to engage in a discussion from varied on- and off-campus locations. The focus group was recorded, with a transcript generated from the recording for later analysis. The focus group was co-facilitated by the first two authors of this manuscript.

During the focus group, participants were asked to speak about their experiences teaching the redesigned courses. Additional questions were generated during the focus group in response to information shared by participants that invited new and different avenues of questioning. All questions centered on participants’ use of redesigned course elements and their perceptions of the experience of teaching a redesigned course.

Data Analysis

Typical of CAE, each participant took part in the analysis of data and determination of outcomes for the study. The first two authors of this study convened a planning meeting with participants to determine the processes that would be used to analyze data that had been collected. The group reached consensus with the following plan:

1. Each participant individually reads all individual participant reflections and the transcript from the focus group session. Participants were asked to generate a list of common themes that emerged from their reading of reflection and focus group data. A video recording of the focus group was also available to participants as they thought about common themes within their ideas.

2. The first author consolidated the themes identified by participants, then shared the complete list with all participants.

3. Participants were asked to give feedback on the consolidated themes to combine similar themes and address any outliers (i.e., themes identified by only one participant).

4. The first author used this second round of feedback to create a list of six themes that emerged from this exercise: equitable course content, flexibility in course policies and structure, course design, spillover effect to other courses, instructional adjustment, and COVID effects.

5. The first author then revisited participant reflections and the focus group transcript and ascribed codes to these documents using the six themes identified through the group’s analysis. Through this work, it was determined that three of the themes (instructional adjustment, COVID effects, and flexibility in course policies and structure) could be compressed into one, as they overlapped one another. Table 3 offers an overview of the themes derived from this process.

Equitable Course Content

Although all participants of this study made changes to include more equitable, diverse, and inclusive course content, each participant focused on different aspects of their course materials. Lisa integrated EDIA topics into almost all chapters of her Lifespan Development course, whereas previously she had only discussed EDIA topics in one chapter in the latter part of the semester. Lisa offered, “I was making the efforts to intentionally spread discussions of diversity, equity, and inclusion issues throughout my Lifespan course.” Her students positively responded to this change in her curriculum. Lisa also indicated “I liked that students were able to see issues of diversity, equity, inclusion, culture, etc. as it relates to human development consistently and across topics.”

Teresa integrated EDIA content by discussing various disparities people face with their insurance and risk management situations. For example, Teresa added the discussion topics that engage students with diverse human experiences and explained her intention behind the change, as “some issues in our subject area that are debatable like things within health insurance like work requirements for Medicaid programs or using insurance companies using credit scores, and what that means for people in terms of you know, the disparity that can cause or social security disability benefits things like this that can be debated sort of in the media.” She continued, “these are things that can make people uncomfortable because some of these topics are politically divisive, or people have opinions on them.”

Blake had yet a different change to his class. His fashion design class uses a lot of images, and he gained critical insights into his use of such visuals. Blake said, “I really never thought about the person wearing a skirt in the background being all white,” as he reflected on his lack of awareness of the racial compositions of his instructional materials. Blake’s change was to make all visuals “representative across the board,” and to, in his words, “be more aware of the images I’m using.”

Molly’s course is the only one that has “diversity” as a specific focus. In her class, she did not add EDIA content as her content already reflected the diverse nature of society. However, she approached her EDIA content differently due to the circumstances in society during that particular semester (e.g., her course was taught the fall semester after George Floyd’s death, which heightened racial consciousness in society). Molly said, “My presentation of the course material changed as a result of the greater diversity in my class,” as she tried to respond to the change in the demographics and student interests in her large lecture course. “The need to express the brutality and unfairness of past repressive institutions of slavery and Jim Crow is demanded of me by some students.”

Flexibility to Course Policies, Structure, and Adjustments

During the course redesign workshop, we discussed the importance of flexibility in course policies and structure to accommodate student needs. With the changing demographic of the student body, many students are juggling non-academic responsibilities, such as their family and work obligations, alongside their academic studies (Bok, 2013; McGee, 2015). Additionally, we discussed Universal Design for Learning (UDL) and examined the importance of multiple means of representations, engagement, and actions and expressions (CAST, 2018) in teaching and learning. All participants in this study considered student needs and desires as major situational factors when redesigning their courses, especially in assessment structures, policies, and deadlines.

Lisa designed a new assignment that allowed her students to choose the unit/topics they were most interested in learning about as a topic for their research paper. She aligned the assignment deadline with when the unit/topics were covered in the course. This allowed students to tailor their due dates to explore their own interests in a way that fit their schedules. While about 50 out of 180 of her students chose the last week for convenience, the majority chose topics from throughout the semester, producing projects that Lisa thought “were amazing and [she] saw their creativity and their passion.” Lisa summarized her experience with the redesigned assignment, and she thought, “The idea that you give them the autonomy to choose what they want to study and dig into and figure out the best mechanism for them to share their information, their talents, and their knowledge, I think it worked really well.”

Teresa also gave her students a lot of flexibility by giving them choices of assessments. She gave students a choice for certain assignments and exams. Teresa said, “I was able to give [students] a lot of flexibility by asking if they wanted to give a presentation live or recorded,” and asked, “Do you want to take an oral final exam or a written final exam?” Teresa also designed her course format purposely, indicating that “I am fortunate I can teach a hybrid course, which means we are in the classroom one day a week and they do work on their own the other time, and so when we are together, we do a lot of actionable work.”

Blake also redesigned his assignments to give additional choices. His assignments reflected each individual student and allowed students to have their voice and to “find their voice.” To achieve this, he created a series of research assignments to help them reflect on their own identity and aspirations. The first project required students to explore their own identities based on who they believed they were. Blake felt that doing some exploration would help students discover who they really are through their design projects.

While Lisa, Teresa, and Blake gave their students flexibility in giving choices and opportunities to express individuality, Molly and Blake implemented flexible assignment deadlines across their courses. Molly allowed “seven days after the due date to turn in an assignment with no penalties if the student emailed or otherwise contacted her.” Blake chose to allow students to “email 48 hours before something is due to propose a new deadline,” which was part of the contract he had his students sign at the beginning of the semester. Additionally, he asked his students to discuss “what happens if you don’t keep up with the work” in the contract, which made his students responsible for completing each step of their course assignments/projects while also giving students an opportunity to develop a skill to be responsible for their own work.

The flexibility that the instructors thought would be beneficial for students sometimes created unexpected issues for student learning. Some of the students in Lisa’s class waited till the very last class to complete their assignments, and these assignments were not as good as those of others who completed the assignments earlier. Lisa talked about this issue and said, “a naïve approach of like their interests will guide them when, in reality, it was their schedules that guided them.” Blake and Molly also had similar issues with flexible deadlines and felt that this type of flexibility did not help students succeed in class. These experiences triggered the instructors to explore ways to help the students become more accountable while staying flexible for subsequent semesters when the redesigned courses would be taught again. Lisa made changes to her assignments so that the students could choose a particular unit instead of the chapter, allowing several weeks for students to submit their assignments. Blake plans to return to having firm deadlines, but will allow students the ability to contact him 48 hours before assignment deadlines if there are delays or anticipated problems. Molly also plans to return to having firm deadlines, but will establish a policy that allows students to contact her ahead of deadlines or submit assignments within a grace period after the deadline with a grade penalty. In making these changes, Molly offered, “I feel as though I’ve moved the responsibility onto [my students].” Thus, the instructors all found ways to make their flexible deadlines work in their future classes, with some “tweaks” that will help students develop planning skills to ensure their own success in current and future courses.

Course Design

Another thing that participants focused on during the course design cohort was to identify the most important content for students to learn as a part of their redesigned course. We explored the question: Five years after students leave your course, what should students remember or be able to do? It is similar to a question asked in Backward Course Design (Wiggins & McTighe, 2005, p. 17), “What should students know, understand, and be able to do?” but this asked participants to expand on that notion to question the truly important aspects of their courses, now and in the future, helping to steer away from any desire to “cover” all possible content in a course.

The instructors in the cohort took the idea to heart and engaged students in the truly important aspects of the topics. As Teresa redesigned the course, she asked herself, “What do I really want them to remember? And that really did drive the way that I taught the courses, and I found myself lingering on bigger issues and less on smaller details.” She explained this transformation by saying, “And that really guided a lot of like, okay, these are little things that don’t matter as much, you know, provisions within Medicaid, but I want them to understand what Medicaid is.” Lisa was also cognizant of focusing on the most important concepts by intentionally connecting assessment, learning activities, and materials with learning objectives. Lisa said, “When I talk about the connection between the week’s topic and the learning objectives at the start of the week, I feel like it highlights that everything we are doing has a purpose.” Molly also wanted to have students understand the most important concepts and remove some details from the course. She tried to give various perspectives on civil rights movements and ended up removing some details. She said, referring to the larger concepts of the Civil Rights Movement, “I’ve had to drop a few things…there are these like little details that I’m like they don’t really need that because that’s distracting, I want them to understand this.”

Spillover Effect on Other Courses

The course redesign cohort required all participants to redesign one course to incorporate EDIA practices and make intentional changes to the course that affect their students. While the chosen courses did see significant changes, many members of the group mentioned the spillover of the principles learned in their PD into other classes. Participants reported making changes to syllabi, presentations, homework, due dates, and learning activities as a part of courses that did not undergo a redesign but that needed an infusion of EDIA in one of many aspects of these courses. This may be a natural outcome of a redesign process as participants prepped new courses or adjusted existing courses to meet new social or environmental changes or challenges. These sorts of adaptations would have a logical impact on other courses, as well, in many cases.

Molly found that her redesign spilled over into not only another history course but also a freshman seminar class. The seminar is very different from her history course, in that it had only 25 students, much smaller than her history classes, which have more than 80 students. In this class, she placed more emphasis on the assignment’s value for the students rather than on the due dates. This resulted in a larger percentage of the students in the class completing the class with higher grades than in previous semesters. In a smaller class, Molly found that some EDIA strategies, including group work, activities involving movement around the room, and reading discussions, generally worked better in a small group setting than in a large group. In that smaller class, Molly also changed the focus of the reading material to focus on civic engagement and emphasized the importance of thinking about the intersectionality between gender and homelessness. Molly said, “it was something I had never thought about bringing up in that class until we did this [course redesign]” and it generated “a deeper and a better discussion than we had ever had before with any other class about that article.”

Teresa explained this spillover effect by stating, “I had success with one particular way of doing something in one course, and I will do that in the other course that was not necessarily reinvented with this purpose in mind….What I was doing in the intro course, which is the course I actually reinvented, spilled over into some of the other courses that I teach.”

Impacts on Individual Courses

Specific impacts identified across participants through the themes of equitable course content and course design were somewhat expected. The redesign cohort specifically allowed participants to learn about equitable and inclusive pedagogies, provided time to develop course content that infused EDIA into their courses, and allowed time for feedback from peers and facilitators throughout the process. While the hope is that course instructors who complete this redesign experience will use what they created as part of their professional development, it is not known how many do, as the intention to make changes to a course following participation in professional development might not always be actualized. That said, participants in this study clearly described ways in which they applied the content they learned and created in their course redesign experience. In fact, several participants described extending their work in real time during the fall semester, adding content beyond what was created in the redesign cohort to further infuse EDIA into their fall course design and delivery. Thus, the notion of creating a course that centered on EDIA was not limited to the week-long redesign cohort experience. Rather, participants demonstrated the motivation to continue to make adaptations to their respective courses well into the fall semester. The continued addition of more inclusive readings, discussion of societal topics impacting higher education, and prioritization of content to create opportunities for deep and meaningful learning highlight just a few aspects of course design that changed for participants as a result of their course redesign experience.

Interestingly, several participants specifically mentioned different types of engagement with their students as another outcome of their course redesign experience. Citing examples of asking students what was important to them, providing students with greater autonomy as learners, and engaging students as partners in making decisions about course assignments and such, participants clearly indicated that their interactions with students changed as a result of implementing their redesigned courses. These examples demonstrate that the impacts of the course redesign experience were not limited to course content or design. Rather, participants seemed to foster a broader, more inclusive view of the teaching and learning context that centered students and their voices in new ways, allowing flexibility and engagement that was not present in prior iterations of their redesigned courses.

Impacts Beyond Individual Courses

Participants in this study were part of a PD experience that was explicitly focused on one course. This is how course redesign cohorts are typically structured: an instructor identifies a course that needs to be rethought for any reason and solely focuses on that course as part of their course redesign experience. With that singular focus in mind, it would be understandable to assume that the impacts of a course redesign experience might be limited to the redesigned course itself. However, the participants in this study indicated otherwise. Not only did their PD experience impact the course they redesigned (and the students enrolled in these courses), but they also made numerous changes to other courses they taught to include more aspects of EDIA in their teaching/learning contexts. These changes were observed in courses of different structure (e.g., seminar vs. lecture courses), size (e.g., large vs. small enrollment), and level (e.g., introductory vs. advanced). These results were unanticipated but provide evidence that application of PD has the potential to transcend contexts even in situations where that was not a primary objective for a PD experience.

Future Implications

Understanding the impacts of professional development is often a challenge for teaching and learning centers (TLCs). TLC staff members and leaders frequently ponder ways in which to observe true changes in teaching behaviors or, more elusively, create a tie between PD for teaching and actual student learning. While it would be impossible to scale this study for all PD offered through a TLC, it is undeniable that the use of CAE to engage participants in a specific professional development experience did allow for the identification of several important impacts resulting from engagement in PD. It might be worth the time and effort of TLC staff to engage in a similar exploration of past participants of PD to truly understand the resulting impacts of PD for course instructors and their students. We are not suggesting that this be the primary method of evaluating PD for teaching and learning; however, such an undertaking might provide important information about the impacts of various types of PD experiences (e.g., course redesign, learning communities, single-time workshops) for TLCs that would help inform future programming and unit priorities.