UC Berkeley

While assessments of general social proficiency constructs are numerous in the ASD literature, and in the special education literature more broadly, investigations of the latent structure(s) of vocational-specific social proficiency constructs have yet to receive serious empirical investigation. Furthermore, trends in this literature point to a number of measurement-related issues that include: item/construct incongruity in the development of instruments; inconsistent representations of the role of context in instruments; relatively outdated approaches to instrument design and analysis of item response data; and lack of evidence for the internal structure of measurement variables in arguments for validity. Finally, the notion of learning progressions in the assessment of social proficiency, and potential models of cognition that underpin them, are not generally considered or empirically tested in this literature. Since educators working to develop social proficiency as part of the services provided to secondary students in special education transitioning to employment may benefit from representations of underlying learning progressions to guide instruction, this work seems timely.

This study extends this body of research by addressing the measurement-related issues introduced above and proposing and testing an explicit model of cognition for the measurement of soft skill proficiency that takes advantage of the principles of sound educational measurement embedded within Wilson's (2005) item response modeling approach to constructing measures. Social Evaluative Reasoning (SER) in the workplace is defined as context-specific critical thinking involving appraisal of the effectiveness and appropriateness of employee behavior as it occurs in response to common antecedent conditions in entry-level employment heavy in soft skill demand. Proficiency in SER ability is conceptualized in an Antecedent (A)-Behavior (B)-Consequence (C) formulation-i.e., within a workplace setting, given (A): some amount of available social information (e.g., a customer's verbal/nonverbal social cues), was (B): a target employee's behavioral response, (C): appropriate given the situational context of the workplace and the organizing and directing forces it places upon employee behavior?

Scenarios in a visual narrative format depicted workplace settings characteristic of entry-level employment heavy in soft skill demand. Manipulation of the social complexity of the scenarios was based on the type, frequency, and co-occurrence of three factors: 1) social perceptual unit (SPU) content: basic versus complex emotions; 2) SPU delivery: literal versus figurative language; and 3) Outcome resolution: correct versus incorrect. Analysis of the item response data utilized the Rasch model various extensions thereof.

Results from this study indicated that a two-dimensional conceptualization of the SER construct, in which this higher-order variable was decomposed on the basis of previous research to differentiate between SPU detection (a local processing task: see A above) and evaluative inferencing (EI) abilities (a global processing task: see B and C above) fit the data best. Importantly, when an adequate number of observations within the proposed levels of the SPU and EI constructs were observed, the hypothesized internal structure of the SPU and EI variables found empirical support by the data. Further support of the two-dimensional structure of the higher-order SER variable was provided in the context of the remaining four strands of the American Educational Research Association (AERA)/American Psychological Association (APA)/ National Council on Measurement in Education's (NCME) (2014) testing standards for validity evidence. Practical implications of this research in addition to implications for research practice were also discussed.