AAAI Publications, Second AAAI Conference on Human Computation and Crowdsourcing

Font Size: 
CrowdUtility: A Recommendation System for Crowdsourcing Platforms
Deepthi Chander, Sakyajit Bhattacharya, Elisa Celis, Koustuv Dasgupta, Saraschandra Karanam, Vaibhav Rajan, Avantika Gupta

Last modified: 2014-09-05

Abstract


Crowd workers exhibit varying work patterns, expertise, and quality leading to wide variability in the performance of crowdsourcing platforms. The onus of choosing a suitable platform to post tasks is mostly with the requester, often leading to poor guarantees and unmet requirements due to the dynamism in performance of crowd platforms. Towards this end, we demonstrate CrowdUtility, a statistical modelling based tool for evaluating multiple crowdsourcing platforms and recommending a platform that best suits the requirements of the requester. CrowdUtility uses an online Multi-Armed Bandit framework, to schedule tasks while optimizing platform performance. We demonstrate an end-to end system starting from requirements specification, to platform recommendation, to real-time monitoring.

Keywords


Crowdsourcing;Platform Recommendation;QoS

Full Text: PDF