Personalized and Diverse Task Composition in Crowdsourcing

Maha Alsayasneh, Sihem Amer-Yahia, Eric Gaussier, Vincent Leroy, Julien Pilourdault, Ria Mae Borromeo, Motomichi Toyama, Jean Michel Renders

Research output: Contribution to journalArticle

8 Citations (Scopus)

Abstract

We study task composition in crowdsourcing and the effect of personalization and diversity on performance. A central process in crowdsourcing is task assignment, the mechanism through which workers find tasks. On popular platforms such as AMT, task assignment is facilitated by the ability to sort tasks by creation date or reward amount. Task composition improves task assignment by producing for each worker, a personalized summary of tasks, referred to as a Composite Task. CTs allow workers to quickly find tasks of interest. We propose different ways of producing CTs and formulate an optimization problem that finds for a worker, the most relevant and diverse CTs. We show that workers' experience is greatly improved due to personalization that enforces an adequation of CTs with workers' skills and preferences. We also study various ways of diversifying tasks in each CT. Task diversity is grounded in organization studies that have shown its impact on worker motivation. Our experiments show that while CTs improve task throughput when compared to ranked lists, diversifying tasks contributes to improving outcome quality. More specifically, we show that task throughput is best when CTs contain tasks having similar topics, while requester-based diversity benefits both worker retention and crowdwork quality. More specifically, we show that while task throughput and worker retention are best with ranked lists, crowdwork quality reaches its best with CTs diversified by requesters, thereby confirming that workers look to expose their “good” work to many requesters.

Original languageEnglish
JournalIEEE Transactions on Knowledge and Data Engineering
DOIs
Publication statusAccepted/In press - 2017 Sep 21

    Fingerprint

Keywords

  • Buildings
  • Clustering algorithms
  • Crowdsourcing
  • Crowdsourcing
  • Optimization
  • Task assignment
  • Task composition
  • Task diversity
  • Throughput
  • Urban areas

ASJC Scopus subject areas

  • Information Systems
  • Computer Science Applications
  • Computational Theory and Mathematics

Cite this

Alsayasneh, M., Amer-Yahia, S., Gaussier, E., Leroy, V., Pilourdault, J., Borromeo, R. M., Toyama, M., & Renders, J. M. (Accepted/In press). Personalized and Diverse Task Composition in Crowdsourcing. IEEE Transactions on Knowledge and Data Engineering. https://doi.org/10.1109/TKDE.2017.2755660