UCLA

In social and behavioral sciences, assessment instruments that are made up of multipleitems (e.g., educational tests, clinical scales) are often used to measure latent variables. Item response theory (IRT) has been employed increasingly frequently in the analysis of these assessment data. Respondents in these fields (e.g., students, patients) often belong to two or more than two non-nested groups simultaneously, yielding the so-called item level data with cross-classified structures. The present study is motivated by the need to appropriately model these item-level data with cross-classified structures, which are prevalent in education and allied disciplines such as psychology.

To achieve the research goal, this study introduces a cross-classified multilevel IRTmodel. The proposed model takes into account both item properties and cross-classified structures. This study also proposes to apply the Metropolis-Hastings Robbins-Monro (MH-RM; Cai, 2008a, 2010b, 2010a) algorithm to address the computational complexities in estimating multilevel latent variable models with crossed random effects, of which the proposed model is a special case. Simulation studies are conducted to evaluate the performance of the MH-RM algorithm in terms of estimating the proposed model. The cross-classified multilevel IRT model is also applied to student evaluations of teaching (SET) collected in a large public university.

This study fills the gap in the IRT literature by introducing the cross-classified multilevelIRT model and an associated estimation algorithm. From a substantive perspective, it also improves the current practice in social and behavioral sciences by introducing an approach that properly models item-level data with cross-classified structures.