Journal of Writing Assessment

The use of reflective essays has become a key artifact of outcome-based writing assessment in the field of writing studies (White, 2005). However, scoring reflective essays may be influenced by textual features irrelevant to most outcomes and assessment rubrics.  Two problematic features are teacher-pleasing, which Yancey (1996) called the “schmooze factor,” and student-fashioning, which Miura (2018) related to identity formation. In this article, we present two mixed methods studies to examine the effects of these particular textual features on the direct assessment of first-year writing (FYW) reflective essays. In the first pilot study, we identified four textual features relevant to teacher-pleasing and student-fashioning. In the second quasi-experimental study, we created a sample of FYW essays with and without these features. Two assessment teams then scored the essays in order to determine whether these features had statistically significant effects on assessment scores. The empirical results of these linked studies indicate these features did not have significant effects in a direct assessment of FYW reflective essays. However, in the discussion and conclusion, we argue our mixed methods approach offers a feasible and efficient set of research methods to examine specific textual features in the direct assessments of student writing. Keywords: writing assessment, first year writing (FYW), reflection, reflective essays, mixed methods