The influence of crowd type and task complexity on crowdsourced work quality

Ria Mae Borromeo, Thomas Laurent, Motomichi Toyama

研究成果: Conference contribution

3 被引用数 (Scopus)

抄録

As the use of crowdsourcing spreads, the need to ensure the quality of crowdsourced work is magnified. While quality control in crowdsourcing has been widely studied, established mechanisms may still be improved to take into account other factors that affect quality. However, since crowdsourcing relies on humans, it is difficult to identify and consider all factors affecting quality. In this study, we conduct an initial investigation on the effect of crowd type and task complexity on work quality by crowdsourcing a simple and more complex version of a data extraction task to paid and unpaid crowds. We then measure the quality of the results in terms of its similarity to a gold standard data set. Our experiments show that the unpaid crowd produces results of high quality regardless of the type of task while the paid crowd yields better results in simple tasks. We intend to extend our work to integrate existing quality con-trol mechanisms and perform more experiments with more varied crowd members.

本文言語English
ホスト出版物のタイトルProceedings of the 20th International Database Engineering and Applications Symposium, IDEAS 2016
編集者Bipin C. Desai, Evan Desai
出版社Association for Computing Machinery
ページ70-76
ページ数7
ISBN(電子版)9781450341189
DOI
出版ステータスPublished - 2016 7月 11
イベント20th International Database Engineering and Applications Symposium, IDEAS 2016 - Montreal, Canada
継続期間: 2016 7月 112016 7月 13

出版物シリーズ

名前ACM International Conference Proceeding Series
11-13-July-2016

Other

Other20th International Database Engineering and Applications Symposium, IDEAS 2016
国/地域Canada
CityMontreal
Period16/7/1116/7/13

ASJC Scopus subject areas

  • ソフトウェア
  • 人間とコンピュータの相互作用
  • コンピュータ ビジョンおよびパターン認識
  • コンピュータ ネットワークおよび通信

フィンガープリント

「The influence of crowd type and task complexity on crowdsourced work quality」の研究トピックを掘り下げます。これらがまとまってユニークなフィンガープリントを構成します。

引用スタイル