AAAI Publications, First AAAI Conference on Human Computation and Crowdsourcing

Font Size: 
A Ground Truth Inference Model for Ordinal Crowd-Sourced Labels Using Hard Assignment Expectation Maximization
Siamak Faridani, Georg Buscher, Ya Xu

Last modified: 2013-11-03


We propose an iterative approach for inferring a ground truth value of an item from judgments collected form on-line workers. The method is specifically designed for cases in which the collected labels are ordinal. Our algorithm works by iteratively solving a hard-assignment EM model and later calculating one final expected value after the convergence of the EM procedure. This algorithm does not require any parameter tuning and can serve as turnkey algorithm for aggregating categorical and ordinal judgments.


crowdsourcing; judgement aggregation; Expectation Maximization;

Full Text: PDF