This article re-examines the research into placement that William L. Smith did at the University of Pittsburgh during the 1980s and 1990s by situating Smith's work within the larger context of educational measurement theories, placement testing, and holistic scoring. I present the series of research studies that Smith conducted into Pitt's placement test as a case study in validation inquiry, arguing that his approach serves as a model for those who direct writing assessments. The implications of Smith's research reach beyond placement into first year composition: by approaching local writing assessment needs as Smith did, writing assessment professionals not only can create more effective assessments, but they also can contribute significantly to assessment theory.

This essay provides an overview of the research and scholarship on reliability in college writing assessment from the author's perspective as a composition and rhetoric scholar. It argues for reframing reliability by drawing on traditions from fields of college composition and educational measurement with the goal of developing a more productive discussion about reliability as we work toward a unified field of writing assessment. In making this argument, the author uses the concept of framing to argue that writing assessment scholars should develop a shared understanding of reliability. The shared understanding begins with the values—such as accuracy, consistency, fairness, responsibility, and meaningfulness—that we have in common with others, including psychometricians and measurement specialists, instead of focusing on the methods. Traditionally, reliability has been framed by statistical methods and calculations associated with positivist science although psychometric theory has moved beyond this perspective. Over time, the author argues, if we can shift the frame associated with reliability, we can develop methods to support assessments that lead to improvement of teaching and learning.

Volume 6 of the Journal of Writing Assessment, our second as co-editors and the third as an online open-access journal, is complete. Looking back at the five articles we published, important themes emerged related to the impact of digital technologies on writing assessment and the connections between rubrics and raters.

Traditionally, editors write an introduction to each issue of a journal, linking articles, identifying themes, and explaining what unifies the volume. However, because our volumes evolve organically with articles being published online as they are deemed ready through the peer-review process, that traditional approach doesn’t work. Instead, we have opted for an Afterword, an opportunity to look back, to see what links and themes have emerged in the volume.

Welcome to the first installment of an annotated bibliography on writing assessment. Over the next several issues, we will be publishing different sections of the bibliography such as Theory, Classroom Response, and Portfolios. Our intent is to provide a resource for all those who work in writing assessment (K-college) or are looking for writing assessment scholarship. In categorizing and sectioning the scholarship, we have used theories and practices of writing assessment instead of educational level because we see a need for more crossover, especially between K-12 and college, in research, theory, and practice.

In this, our third installment of the bibliography on assessment, we survey the second half of the literature on reliability and validity. The works we annotate focus primarily on the theoretical and technical definitions of reliability and validity--and in particular, on the relationship between the two concepts. We summarize psychometric scholarship that explains, defines, and theorizes reliability and validity in general and within the context of writing assessment. Later installments of the bibliography will focus on specific sorts of assessment practices and occasions, such as portfolios, placement assessments, and program assessment--all practices for which successful implementation depends on an understanding of reliability and validity.

This article reports on a study of the impact of the form of high-stakes, state-mandated writing tests on high school curricula and teaching practices. Through surveys and focus group interviews of high school English teachers, we addressed two main questions: How do different kinds of high-stakes, statewide assessments impact writing curriculum and instruction in secondary schools? and What are teachers' views of the impact of different kinds of high-stakes tests? We conducted our study in three states--California, Georgia, and Kentucky--each with a different type of writing test at the time of the study: multiple choice, timed impromptu, and portfolio, respectively. The survey results contribute to the growing body of research that indicates the forms of writing tests influence what teachers teach and how they teach it. This influence was complex with significant differences across the three states in the types of assignments, the length of assignments, number of drafts, and the amount of time allowed for assignments. Our results also indicated that the form of high-stakes writing tests also impacts teacher morale and attitudes.

This annotated bibliography focuses on issues surrounding minorities and writing assessment, including issues associated with various ethnic groups as well as those issues associated with gender studies and with minorities in special education. In addition to reporting on minorities and testing consequences, several selections make recommendations for revising or constructing testing instruments to address some of the special issues minorities face with classroom and large-scale assessments. Several selections also pose important questions that classroom instructors, school administrators, and writing assessment specialists might ask before conducting assessments that involve minority students. Over the past 30 years, studies of SAT, ACT, and other high-stakes tests report that test scores continue to lag behind for certain minority groups. Several selections that we annotate below offer reasons for these differences and fresh insights and analyses into trends for minority test scores.

In this, our second installment of the bibliography on assessment, we survey the literature on reliability and validity, the first of a two-part series that will continue in the next issue of JWA. The works we annotate focus primarily on the theoretical and technical definitions of reliability and validity--and in particular, on the relationship between the two concepts. We summarize psychometric scholarship that explains, defines, and theorizes reliability and validity in general and within the context of writing assessment. Later installments of the bibliography will focus on specific sorts of assessment practices and occasions, such as portfolios, placement assessments, and program assessment--all practices for which successful implementation depends on an understanding of reliability and validity.