UC Berkeley

In 2011, the American Evaluation Association published the Public Statement on Cultural Competence in Evaluation, highlighting the need for evaluators to take a culturally competent stance. While some may view this charge as primarily applicable to qualitative methods, evaluators with quantitative skillsets are critical contributors. Advances in quantitative techniques allow for statistical modeling to address nuanced questions appropriate to a culturally competent evaluation design; however, the link between cultural competence and statistical modeling remains unclear.

Prior research has discussed the need for an inquiry into if and how quantitative methods can help evaluators better respond to and engage with culture (Chouinard and Cousins, 2009). Chapter One of this dissertation responds to this need by reviewing 110 evaluation cases demonstrating culturally competent approaches and synthesizing their prevalence and use of quantitative methods. Findings discuss the predominant methods and purposes for integrating quantitative methods with culturally competent approaches. Furthermore, findings illustrate key themes for aligning quantitative practices with culturally competent approaches.

One of the key themes from Chapter One highlighted the need for quantitative evaluation measures to better align with the cultural context of an evaluation. Therefore, Chapter Two outlines the Bear Assessment System process for creating, validating, and calibrating quantitative measures in a manner that complements a culturally competent evaluation approach. This process is explained through reflecting on the case of UC Berkeley’s Athletic Study Center evaluation in which quantitative measures of sense of belonging and self-reliance were created for both formative and summative purposes.

While Chapter Two explained how to create, validate, and calibrate quantitative measures, the literature still lacks a clear example of how to analyze measures to reflect cultural competence. Chapter Three extends the evaluation case presented in Chapter Two to demonstrate how an evaluator can apply the Latent Growth Item Response model (LG-IRM) to analyze longitudinal data with both statistical rigor and cultural competence.