This study applies the Standards for Educational and Psychological Testing to appraise validity evidence for the interpretation of test scores that are used in experiential education, adventure education, and out-of-school time programs. Specifically, Rasch item response modeling is applied as a primary tool to examine validity evidence for the Life Effectiveness Questionnaire (LEQ). Hypotheses and assumptions about test content and test structure are evaluated using technically calibrated responses to items as empirical evidence. This evidence, produced in the form of Wright maps, allows for a combined focus on the qualitative, substantive meaning of the test scores, as well as their technical, statistical properties.

The partial credit model fit better than the rating scale model for the composite unidimensional and multidimensional models (p>0.05) of the LEQ, and for seven of the eight consecutive unidimensional models of LEQ subscales (p>0.05). DIF analyses found no meaningful statistical evidence of construct-irrelevant variance by gender, age, or voluntary status. Analyses of person to item distributions found a mismatch between the intended meaning of item content and empirical item difficulties, so that the test developers' initial hypothesis about item content was not supported. Evidence from Wright maps indicates no order or sequencing of LEQ test content, and also that items oversample from one easy level of the Life Effectiveness continuum. These findings suggest that item content does not adequately represent the Life Effectiveness construct.

Evidence from Wright maps indicates that respondents did not interpret and differentiate between response categories equivalently within items or across subdomains, so that assumptions of interval, Likert scaling that are the basis of the averaged scores for the LEQ are not supported. Wright maps provided evidence of good respondent-to-item threshold distribution for the composite LEQ. However, item thresholds are skewed to the lower levels of life effectiveness, so that standard errors are higher for respondents with higher ability estimates. Despite reasonable reliability coefficients, sizable standard errors and large confidence intervals associated with the LEQ subscales, in particular, as well as for high ability respondents in the composite LEQ, caution against over-confidence in using student ability estimates to make claims about change.

This dissertation consists of three papers on a central topic: the application of scale type theory to the Rasch and 2PL models. Each paper uses a different framework or set of frameworks for defining a typology of scales.

In the first paper, I begin the analysis of scale types through the Stevens (1946) typology. I introduce the notion of difference scales using a formalization of this typology. In applying this formalization to the Rasch and 2PL models, I discuss alternate paradigms for conceptualizing the restrictive assumptions of the Rasch model.

In the second paper, I apply the Suppes (1958) typology to the two IRT models. The conclusions here echo and reinforce those of the first chapter. In the second half of this paper, I examine other perspectives on connecting scale type theory and the Rasch model, focusing on the case of additive conjoint measurement theory.

The first two papers consider the scale types of the odds and logit forms of the models separately. In the third paper, I look at typologies in which the model form is not a factor. The first of these is the axiomatic system of Hölder (1901), which identifies two types of quantities: magnitudes, and points on a line. The second is the homogeneity-uniqueness typology of Narens and Luce (1986), which classifies scales by the types of choices that are possible during the process of numeral assignment.

Together, the three papers form an argument for considering psychometric properties behaving as predicted by the Rasch model as having the scale type of ratio scale attributes, while those that behave as predicted by the 2PL model have the scale type of interval scale attributes.

Parental burnout, characterized by overwhelming exhaustion related to parenting demands, represents a significant health concern across diverse cultural contexts. This study investigates the phenomenon's global prevalence, contributing factors, and consequences, emphasizing Differential Item Functioning (DIF) within the Parental Burnout Assessment (PBA) across genders and countries. Utilizing a robust dataset, the research explores how cultural, social, and gender-specific factors influence parental burnout experiences, highlighting the need for gender-sensitive approaches and interventions tailored to varied cultural norms.

The methodology employs a secondary analysis of the comprehensive dataset by Roskam et al. (2021), examining gender and country invariance in responses to the PBA. Through the application of the Multidimensional Partial Credit Model (MPCM), the study reveals nuanced patterns of parental burnout, offering insights into the complex interplay between societal norms, cultural expectations, and parental roles. The findings underscore significant variations in the manifestation of parental burnout across countries, with notable differences in the experiences of mothers and fathers.

This dissertation contributes to a deeper understanding of parental burnout as a multidimensional and culturally influenced phenomenon. By identifying and analyzing DIF across genders and countries, the research underscores the importance of culturally and gender-sensitive assessments and interventions. The study's insights into the global nuances of parental burnout are vital for developing targeted support systems, fostering resilience among families worldwide, and promoting healthier family dynamics in the face of parenting challenges.

Motivated by the recent discourses on approaches to combine generalizability theory (GT) and item response theory (IRT), this study suggests an approach that answers to some of the issues raised by previous research on combining GT and IRT. The main idea of the proposed approach is to recognize that IRT models can be written in terms of a latent continuous response and that a classic IRT model can be modeled directly using standard GT with items as a fixed facet. Once this is recognized, treating items as a random facet or considering other facets such as raters, become relatively straightforward extensions. The resulting models are logistic mixed models that contain the parameter of interest: the variance components needed for the generalizability and dependability coefficients. The models can be estimated by straightforward ML estimation. Extensive simulation studies were conducted to evaluate the performance of the proposed approach under various measurement situations. The results suggested that the proposed approach gives overall accurate results, in particular, when estimating the generalizability coefficients. The use of the proposed method is illustrated with large-scale data sets from classroom assessment (the Carbon Cycle 2008-2009 data set) and standardized tests (the PISA 2009 U.S. Science data set). The empirical results were presented and interpreted in the context of the G study and the D study in GT as well as a Wright Map in IRT. The results demonstrated the flexibility of the proposed approach with respect to incorporating extra complications in measurement situations (e.g., multidimensionality, polytomous responses) and further explanatory variables (e.g., rater facet).