Challenges in rating signed production: A mixed-methods study of a Swiss German Sign Language form-recall vocabulary test

Aaron Olaf Batty, Tobias Haug, Sarah Ebling, Katja Tissi, Sandra Sidler-Miserez

Research output: Contribution to journalArticlepeer-review

Abstract

Sign languages present particular challenges to language assessors in relation to variation in signs, weakly defined citation forms, and a general lack of standard-setting work even in long-established measures of productive sign proficiency. The present article addresses and explores these issues via a mixed-methods study of a human-rated form-recall sign vocabulary test of 98 signs for beginning adult learners of Swiss German Sign Language (DSGS), using post-test qualitative rater interviews to inform interpretation of the results of quantitative analysis of the test ratings using many-facets Rasch measurement. Significant differences between two expert raters were observed on three signs. The follow-up interview revealed disagreement on the criterion of correctness, despite the raters’ involvement in the development of the base lexicon of signs. The findings highlight the challenges of using human ratings to assess the production not only of sign language vocabulary, but of minority languages generally, and underscore the need for greater effort expended on the standardization of sign language assessment.

Original languageEnglish
JournalLanguage Testing
DOIs
Publication statusAccepted/In press - 2022

Keywords

  • Many-facets Rasch measurement
  • rater behavior
  • sign-language assessment
  • Swiss German Sign Language
  • vocabulary assessment

ASJC Scopus subject areas

  • Language and Linguistics
  • Social Sciences (miscellaneous)
  • Linguistics and Language

Fingerprint

Dive into the research topics of 'Challenges in rating signed production: A mixed-methods study of a Swiss German Sign Language form-recall vocabulary test'. Together they form a unique fingerprint.

Cite this