Building a better rubric: Mixed methods rubric revision

Gerriet Janssen, Valerie Meier, Jonathan Trace

Research output: Contribution to journalArticle

14 Citations (Scopus)

Abstract

Because rubrics are the foundation of a rater's scoring process, principled rubric use requires systematic review as rubrics are adopted and adapted (Crusan, 2010, p. 72) into different local contexts. However, detailed accounts of rubric adaptations are somewhat rare. This article presents a mixed-methods (Brown, 2015) study assessing the functioning of a well-known rubric (Jacobs, Zinkgraf, Wormuth, Hartfiel, & Hugley 1981, p. 30) according to both Rasch measurement and profile analysis (n = 524), which were respectively used to analyze the scale structure and then to describe how well the rubric was classifying examinees. Upon finding that there were concerns about a lack of distinction within the rubric's scale structure, the authors decided to adapt this rubric according to theoretical and empirical criteria. The resulting scale structure was then piloted by two program outsiders and analyzed again according to Rasch measurement, placement being measured by profile analysis (n = 80). While the revised rubric can continue to be fine-tuned, this study describes how one research team developed an ongoing rubric analysis, something that these authors recommend be developed more regularly in other contexts that use high-stakes performance assessment.

Original languageEnglish
Pages (from-to)51-66
Number of pages16
JournalAssessing Writing
Volume26
DOIs
Publication statusPublished - 2015 Oct 1
Externally publishedYes

    Fingerprint

Keywords

  • Academic writing
  • Mixed-methods
  • Profile analysis
  • Rasch measurement
  • Rubrics

ASJC Scopus subject areas

  • Language and Linguistics
  • Education
  • Linguistics and Language

Cite this