A comparison of video- and audio-mediated listening tests with many-facet Rasch modeling and differential distractor functioning

Research output: Contribution to journalArticle

12 Citations (Scopus)

Abstract

The rise in the affordability of quality video production equipment has resulted in increased interest in video-mediated tests of foreign language listening comprehension. Although research on such tests has continued fairly steadily since the early 1980s, studies have relied on analyses of raw scores, despite the growing prevalence of item response theory in the field of language testing as a whole. The present study addresses this gap by comparing data from identical, counter-balanced multiple-choice listening test forms employing three text types (monologue, conversation, and lecture) administered to 164 university students of English in Japan. Data were analyzed via many-facet Rasch modeling to compare the difficulties of the audio and video formats; to investigate interactions between format and text-type, and format and proficiency level; and to identify specific items biased toward one or the other format. Finally, items displaying such differences were subjected to differential distractor functioning analyses. No interactions between format and text-type, or format and proficiency level, were observed. Four items were discovered displaying format-based differences in difficulty, two of which were found to correspond to possible acting anomalies in the videos. The author argues for further work focusing on item-level interactions with test format.

Original languageEnglish
Pages (from-to)3-20
Number of pages18
JournalLanguage Testing
Volume32
Issue number1
DOIs
Publication statusPublished - 2015 Jan 30

Keywords

  • differential distractor functioning
  • language assessment
  • listening assessment
  • many-facet Rasch measurement
  • nonverbal communication
  • video listening test

ASJC Scopus subject areas

  • Language and Linguistics
  • Social Sciences (miscellaneous)
  • Linguistics and Language

Fingerprint Dive into the research topics of 'A comparison of video- and audio-mediated listening tests with many-facet Rasch modeling and differential distractor functioning'. Together they form a unique fingerprint.

  • Cite this