AAAI Publications, Third AAAI Conference on Human Computation and Crowdsourcing

Font Size: 
Flexible Reward Plans to Elicit Truthful Predictions in Crowdsourcing
Yuko Sakurai, Satoshi Oyama, Masato Shinoda, Makoto Yokoo

Last modified: 2015-09-23


We develop a flexible reward plan to elicit truthful predictive probability distribution over a set of uncertain events from workers.  In our reward plan, the principal can assign rewards for incorrect predictions according to her similarity between events.  In the spherical proper scoring rule, a worker's expected utility is represented as the inner product of her truthful predictive probability and her declared probability. We generalize the inner product by introducing a reward matrix that defines a reward for each prediction-outcome pair. We show that if the reward matrix is symmetric and positive definite, the spherical proper scoring rule guarantees the maximization of a worker's expected utility when she truthfully declares her prediction.

Full Text: PDF