Center for Studies in Higher Education

Validity researchers typically work with nonrandom samples, membership in which depends in part on the exam score being investigated. In a study of the SAT's validity for freshman GPA at a particular college, for example, FGPA is not observed for the entire pool of potential applicants, but only for those who are admitted and enroll. This sample selection biases validity estimates. Corrections for restriction of range remedy the problem only when the exam score is the sole determinant of selection, and even then do not permit consistent estimation of the exam's incremental validity. Regression omitted variables results motivate a proposed validity estimator that is consistent whenever the determinants of selection are observed. An algorithm is suggested for calculation of these "robust" validity coefficients, and used to estimate the SAT's validity at the University of California. The usual validity estimates are shown to be substantially biased, in the hypothesized directions, by admissions-induced sample selection.