Multilevel Factor Analysis and Student Ratings of Instructional Practice
- Author(s): Schweig, Jonathan David
- Advisor(s): Martinez, Jose Felipe
- et al.
Student surveys of classroom climate can provide teachers and researchers with valuable information about instruction. These surveys are becoming a critical component in policy efforts to assess and improve teaching. Advocates note that students are natural observers of their classroom environments, have extensive and rich knowledge of their teachers, and that student ratings of teacher practice can be predictive of important outcomes, such as student academic and socio-emotional development. Inferences about a teacher's instructional practice are often based on aggregated student survey responses, and a key step in assessing the appropriate uses of the information collected from student surveys is to understand the dimensions of classroom climate or instructional practice that are discernible when looking at student responses aggregated by classroom.
This dissertation proposes a new approach for exploring the dimensionality of aggregated student ratings. This approach also has the potential to provide validity evidence supporting the use of student surveys as measures of instructional practice in both formative and summative evaluation. Specifically, this dissertation applies a non-parametric cluster bootstrap technique to a multilevel covariance structure analysis framework that allows researchers to evaluate and investigate psychometric models when data is collected from students and teachers are the objects of measurement. This approach is extended to applications where teachers are clustered within schools. The cluster bootstrap technique is demonstrated on a realistic dataset to illustrate how the methods may be used to investigate the dimensions of teacher professional practice that are discernible on a state-wide student survey of instructional practice.
The results of this dissertation demonstrate that the proposed cluster bootstrap technique can be used in conjunction with maximum likelihood estimation to yield accurate parameter estimates, and that for sufficiently large sample sizes, test statistics and standard errors based on the cluster bootstrap technique will yield valid inferences about the psychometric properties of aggregated survey responses.