AAAI Publications, First AAAI Conference on Human Computation and Crowdsourcing

Font Size: 
HiveMind: Tuning Crowd Response with a Single Value
Preetjot Singh, Walter S. Lasecki, Paulo Barelli, Jeffrey P. Bigham

Last modified: 2013-11-03


One common problem plaguing crowdsourcing tasks is tuning the set of worker responses: Depending on task requirements, requesters may want a large set of rich and varied worker responses (typically in subjective evaluation tasks) or a more convergent response-set (typically for more objective tasks such as fact-checking). This problem is especially salient in tasks that combine workers’ responses to present a single output: Divergence in these settings could either add richness and complexity to the unified answer, or noise. In this paper we present HiveMind, a system of methods that allow requesters to tune different levels of convergence in worker participation for different tasks simply by adjusting the value of one variable.

Full Text: PDF