Automatic vs. crowdsourced sentiment analysis

Ria Mae Borromeo, Motomichi Toyama

Research output: Chapter in Book/Report/Conference proceedingConference contribution

8 Citations (Scopus)

Abstract

Due to the amount of work needed in manual sentiment analysis of written texts, techniques in automatic sentiment analysis have been widely studied. However, compared to manual sentiment analysis, the accuracy of automatic systems range only from low to medium. In this study, we solve a sentiment analysis problem by crowdsourcing. Crowdsourcing is a problem solving approach that uses the cognitive power of people to achieve specific computational goals. It is implemented through an online platform, which can either be paid or volunteer-based. We deploy crowdsourcing applications in paid and volunteer-based platforms to classify teaching evaluation comments from students. We present a comparison of the results produced by crowdsourcing, manual sentiment analysis, and an existing automatic sentiment analysis system. Our findings show that the crowdsourced sentiment analysis in both paid and volunteer-based platforms are considerably more accurate than the automatic sentiment analysis algorithm but still fail to achieve high accuracy compared to the manual method. To improve accuracy, the effect of increasing the size of the crowd could be explored in the future.

Original languageEnglish
Title of host publicationACM International Conference Proceeding Series
PublisherAssociation for Computing Machinery
Pages90-95
Number of pages6
EditionCONFCODENUMBER
DOIs
Publication statusPublished - 2015 Jul 13
Event19th International Database Engineering and Applications Symposium, IDEAS 2015 - Yokohama, Japan
Duration: 2015 Jul 132015 Jul 15

Other

Other19th International Database Engineering and Applications Symposium, IDEAS 2015
CountryJapan
CityYokohama
Period15/7/1315/7/15

    Fingerprint

Keywords

  • Crowdsourcing
  • Sentiment analysis
  • Text tagging

ASJC Scopus subject areas

  • Human-Computer Interaction
  • Computer Networks and Communications
  • Computer Vision and Pattern Recognition
  • Software

Cite this

Borromeo, R. M., & Toyama, M. (2015). Automatic vs. crowdsourced sentiment analysis. In ACM International Conference Proceeding Series (CONFCODENUMBER ed., pp. 90-95). [2790761] Association for Computing Machinery. https://doi.org/10.1145/2790755.2790761