Journal of Writing Assessment

Editors' introduction to the Special Issue on a Theory of Ethics for Writing Assessment.

In this introductory article, I set the stage for the arguments that follow in each of the contributions to this special issue. First, I critically examine the three pillars of the current Standards--fairness, validity, and reliability--exploring briefly how on their own each concept is insufficient to guiding ethical practice. Then I briefly examine the Standards themselves highlighting their limitations in guiding ethical practice. Finally, I provide a brief introduction to the various dimensions of the theory of ethics we are developing in this special issue.

This paper proposes a theory of ethics for writing assessment. Based on a definition of fairness as the identification of opportunity structures created through maximum construct representation under conditions of constraint--and the toleration of constraint only to the extent to which benefits are realized for the least advantaged--the theory is expressed in terms of its tradition, boundary, order, and foundation. To examine the force of the theory, a thought experiment demonstrating action based on the theory is offered so that its weaknesses and strengths are identified. Intended for the research specialization of writing assessment, the theory has generalization implications for the field of writing studies.

In my introduction to this special issue, I highlighted the insufficiency of key measurement concepts--fairness, validity, and reliability--in guiding the design and implementation of writing assessments. I proposed that the concept of ethics provides a more complete framework for guiding assessment design and use. This article advances the philosophical foundation for our theory of ethics articulated by Elliot (this issue). Starting with fairness as first principle, it examines how safety and risk can be addressed through the application of an integrated design and appraisal framework (IDAF) for writing assessment tools. The paper is structured around two case studies set in Alberta, Canada. Case Study 1 applies Kane's (2013) IUA model of validation to an appraisal--Alberta's English 30-1 (grade 12 academic English) diploma exam program--highlighting in the process the limitations in contemporary validity theory. Case Study 2 examines an assessment design project I am currently undertaking in partnership with 8 English language arts teachers in southern Alberta. This case study examines how the IDAF supports ethical assessment design and appraisal.

The Civil Rights Act of 1964 has served as an influential legal framework for addressing intentional (disparate treatment) and unintentional (disparate impact) discrimination. While philosophical and methodological discussions of Title VI and Title VII are well articulated in the legal scholarship, the disparate impact approach--a method for evaluating unintended racialized differences in outcomes resulting from facially neutral policies or practices--remains an underutilized conceptual and methodological framework in assessment literature. In this article, we argue that the burden-shifting heuristic used by entities such as the Office for Civil Rights to redress disparate impact is a valuable approach in evaluating fairness of writing assessment practices. In demonstrating an application of the burden-shifting approach at one university writing program, we discuss the value of the proposed integrative framework and point to remaining questions regarding sampling concerns--group identification, group stability, and intersectionality.

The other articles in this issue (Elliot, Slomp, Poe & Cogan, and Cushman) gather resources from the disciplines of philosophy, validity theory, law, and cultural studies to forge a new praxis of ethical writing assessment. My contribution takes a different approach to evaluative ethics, an approach we could term political. On the current political scene of writing assessment in the United States,[1] one heavy ethical burden lies neglected: the adverse educational consequences of mass-market standardized testing, especially the testing of writing ability. This article examines failures by two prominent testing corporations to take up their ethical responsibilities to protect the "primary good" (Rawls) of educational quality. One corporation's failure appears at the end of an evidently serious and good-faith effort to engage its responsibilities; the other corporation's failure results from its apparent lack of effort and commitment to that task. I explain both failures by pointing to the phenomenon of "structured ethical blindness", in which truths that stand in the way of someone's pursuit of profit are difficult or impossible for that person to see and to take seriously. My goal is to understand structured ethical blindness in mass-market writing assessment and to suggest how our profession and our society might best respond to this phenomenon. Based on my analyses, I conclude that testing corporations cannot fairly or realistically be expected to run their businesses with careful attention to the educational consequences of their products. Their unavoidable self-interest prevents them from seeing those consequences clearly. I therefore propose federal regulation and oversight as the only apparently functional mechanism by which to counterbalance testing corporations' pursuits of private profit with the U.S. public's right and responsibility to protect the quality of students' educations.

Legacies of imperialist thought permeate understandings and uses of validity. This essay discusses the ways in which the concept of validity creates the colonial difference as it maintains social, epistemic, and linguistic hierarchies. Recent scholarship in writing assessment, particularly in this special issue, is considered in light of this critique as it offers ethical and methodological alternatives to help address issues of equity, representation, and fairness. Though attempts to revise the Standards to address issues of fairness have been made by the American Educational Research Association [AERA], American Psychological Association [APA], & National Council on Measurement in Education [NCME], these remain partial and incomplete. I propose a decolonial option to modernist notions of validity, one that asks scholars of assessment and test designers to dwell in the borders of the colonial difference that validity, as both hermeneutic and instrument, has helped to create and ensure. Such a decolonial approach would see validity evidence tools, not as a way to maintain, protect, conform to, confirm, and authorize the current systems of assessment and knowledge making, but rather as one way to better understand difference in and on its own terms.

We hope this special issue adds to the body of knowledge created by the writing studies community with respect to the opportunities that can be created when assessment is seen in terms of the creation of opportunity structure. This hope is accompanied by a reminder of our strength as we annually encounter approximately 48.9 million students in public elementary and secondary schools 20.6 million students in postsecondary institutions (Snyder & Dillow, 2015). Our influence is remarkable as we touch the lives of many, one student at a time.