Extensions and Applications of Item Explanatory Models to Polytomous Data in Item Response Theory
- Author(s): Kim, Jinho
- Advisor(s): Wilson, Mark
- et al.
Polytomous item explanatory models aim to provide informative feedback for improving item design and generation, test development, and educational assessment so that they can help improve teachers’ instruction and students’ learning ultimately. In order to demonstrate methodological advantages and practical implications of these models, this dissertation addresses extensions and applications of item explanatory models to polytomous data, using the Linear Logistic Test Model (LLTM; Fischer, 1973) approach, in the context of explanatory measurement (De Boeck & Wilson, 2004). The three papers in this dissertation are intended to be separately publishable, but they share the common theme of explanatory and predictive inferences from the ordered-category item responses.
The first paper in Chapter 2 discusses the applicability of the Many-Facet Rasch Model (MFRM; Linacre, 1989) to polytomous item explanatory models, considering categorical item properties as facets or sub-facets within the item facet. To demonstrate practical applications of the MFRM-based item explanatory model, two empirical studies investigate how item properties explain and predict the overall item difficulties in the Carbon Cycle assessment data and in the Social Evaluative Reasoning in the workplace data respectively. The results suggest that the MFRM-based item explanatory models enable to check and validate underlying hypotheses applied to the item design and construction as well as to develop a new item in a scientific and systematic manner rather than an intuitive manner. The results serve as helpful feedback for enhancing quality of the assessment and instrument development.
The second paper in Chapter 3 investigates polytomous item explanatory models based on the adjacent-categories logits under the multivariate generalized linear mixed modeling framework. Building on the original ideas of the MFRM and the Linear Partial Credit Model (LPCM; Fischer & Ponocny, 1994), polytomous Rasch family models are extended to two item explanatory versions using the MFRM and the LPCM approaches. To demonstrate the practical differences between the two polytomous item explanatory approaches, two empirical studies in this paper examine how item properties explain and predict the overall item difficulties or the step difficulties each in the Carbon Cycle assessment data and in the verbal aggression data. The results suggest that the two polytomous item explanatory models are methodologically and practically different in terms of the target difficulty parameters of polytomous items which are explained by item properties, the types of item properties incorporated in the design matrix, and the types of item property effects.
The third paper in Chapter 4 proposes three polytomous item explanatory models with random item errors, based on the models specified in the second paper. As in the Linear Logistic Test Model with item error (LLTM + ε; Mislevy, 1988; Janssen, Schepers, & Peres, 2004) approach, the proposed polytomous random item effects models can take the uncertainty in explanation and/or the random nature of item parameters into account for polytomous items. For estimation of the proposed models with crossed random effects, available estimation methods are reviewed, and a Bayesian inference is adopted. To examine how the proposed models function in different explanatory conditions, two simulation studies evaluate model comparison and parameter recovery as well as the effects of model misspecification. In addition, two empirical studies demonstrate practical implications and applications of the proposed models to two real data sets, the Carbon Cycle assessment data and the verbal aggression data. The simulation findings suggest that the proposed polytomous item explanatory models with random item errors perform better than the models without item error terms for making accurate statistical inferences, such as estimation or hypothesis testing for the item property effects. The empirical findings indicate that the proposed models outperform the models without random item errors in terms of the goodness-of-fit and reconstructing the step difficulties, and also show that methodological and practical differences between the two polytomous item explanatory approaches.
Keywords: explanatory measurement, item property, item explanatory model, polytomous data, linear logistic test model, many-facet Rasch model, linear partial credit model, linear logistic test model with item error, random item effects, random item error