Journal of Writing Assessment

Because writing assessment’s present is bound to its past (Elliot, 2005; Poe et al., 2018), scholarship has pointed to the need for more critical inquiries of local “assessment ecologies” (Inoue, 2015) to better understand the effects of past injustices (Hammond, 2018; Harms, 2018). In understanding how opportunities are allocated unjustly within processes like articulation, compositionists must be willing to understand how postsecondary ecologies have systematically attempted to deny opportunities for certain student groups. This article does so by examining the 1935 Michigan Committee on the Articulation of High School and College English, a committee in charge of redefining readiness for first-year writing across the state of Michigan, led by Professor Clarence DeWitt Thorpe of the University of Michigan. The work of Thorpe’s committee has been an under examined component of historical assessment ecologies in the Midwest and beyond. Under Thorpe’s guidance, this committee shaped assessment to function across numerous institutional spaces. Utilizing this historical example, this article illustrates that narrow emphases on the ways local ecologies shape opportunities are insufficient for identifying how opportunities are structured. Instead, scholarship should be more attentive in understanding how any given local ecology is situated within a broader system of interinstitutional ecologies that define the boundaries and formations of opportunity within the space.

Keywords:  opportunity, assessment ecology, writing assessment, history, social justice

The use of reflective essays has become a key artifact of outcome-based writing assessment in the field of writing studies (White, 2005). However, scoring reflective essays may be influenced by textual features irrelevant to most outcomes and assessment rubrics.  Two problematic features are teacher-pleasing, which Yancey (1996) called the “schmooze factor,” and student-fashioning, which Miura (2018) related to identity formation. In this article, we present two mixed methods studies to examine the effects of these particular textual features on the direct assessment of first-year writing (FYW) reflective essays. In the first pilot study, we identified four textual features relevant to teacher-pleasing and student-fashioning. In the second quasi-experimental study, we created a sample of FYW essays with and without these features. Two assessment teams then scored the essays in order to determine whether these features had statistically significant effects on assessment scores. The empirical results of these linked studies indicate these features did not have significant effects in a direct assessment of FYW reflective essays. However, in the discussion and conclusion, we argue our mixed methods approach offers a feasible and efficient set of research methods to examine specific textual features in the direct assessments of student writing. Keywords: writing assessment, first year writing (FYW), reflection, reflective essays, mixed methods