UCLA

The Levels of Conceptual Understanding in Statistics (LOCUS) assessment is a standardized test designed to measure conceptual understanding in statistics education. Educational assessment data, such as LOCUS, usually exhibit hierarchical structure, with students nested within teachers, schools, and state. Ignoring this structure can often lead to misidentifying important sources of variation and can lead to misleading interpretations of student performance. This thesis applies hierarchical linear modeling (HLM) and other statistical analyses to the LOCUS dataset in order to examine and quantify teacher variation in exam performance. The thesis begins with an exploratory analysis of the LOCUS dataset. Then, hierarchical linear models are fit separately across ten LOCUS exam forms to determine within-teacher and between-teacher variance components. Null models indicate substantial clustering at the teacher level across all forms. Full models incorporate fixed effects for school year, population of test takers, testing phase, and state in order to evaluate how much between-teacher variation can be explained by observable covariates. Results show that these covariates do account for a substantial portion of teacher-level variance, with state contributing the largest reductions in variance. However, estimated state effects are unstable across forms. After adjustment for observable covariates, residual teacher-level ICCs remain consistent across forms, generally ranging from approximately 0.15 to 0.20. This thesis provides evidence that variation in LOCUS performance does exist at the teacher level, even after accounting for fixed effects. These findings contribute to a better understanding of the factors associated with students' conceptual understanding of statistics over the years of LOCUS administration.