AAAI Publications, Second AAAI Conference on Human Computation and Crowdsourcing

Font Size: 
Crowd-Workers: Aggregating Information Across Turkers to Help Them Find Higher Paying Work
Chris Callison-Burch

Last modified: 2014-09-05


The Mechanical Turk crowdsourcing platform currently fails to provide the most basic piece of information to enable workers to make informed decisions about which tasks to undertake: what is the expected hourly pay?  Mechanical Turk advertises a reward amount per assignment, but does not give any indication of how long each assignment will take.  We have developed a browser plugin that tracks the length of time it takes to complete a task, and a web service that aggregates the information across many workers. Our web service,, allows workers to discovery higher paying work by sorting tasks by estimated hourly rate.


crowdsourcing, fair pay, mechanical turk


Bederson, B. B., and Quinn, A. J. 2011. Web workers unite! addressing challenges of online laborers. In CHI ’11 Ex- tended Abstracts on Human Factors in Computing Systems, CHI EA ’11, 97–106.

Ipeirotis, P. G. 2010a. Analyzing the Mechanical Turk mar- ketplace. In ACM XRDS.

Ipeirotis, P. G. 2010b. Demographics of Mechanical Turk. Technical Report Working paper CeDER-10-01, New York University, Stern School of Business.

Irani, L. C., and Silberman, M. S. 2013. Turkopticon: Inter- rupting worker invisibility in Amazon Mechanical Turk. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’13, 611–620.

Martin, D.; Hanrahan, B. V.; O’Neill, J.; and Gupta, N. 2014. Being a Turker. In Proceedings of The 17th ACM Confer- ence on Computer Supported Cooperative Work and Social Computing (CSCW).

Mason, W., and Watts, D. J. 2009. Financial incentives and the “performance of crowds”. In Proceedings of the Workshop on Human Computation at the International Con- ference on Knowledge Discovery and Data Mining (KDD- HCOMP ’09).

Silberman, M. S.; Ross, J.; Irani, L.; and Tomlinson, B. 2010. Sellers’ problems in human computation markets. In Proceedings of the ACM SIGKDD Workshop on Human Computation, HCOMP ’10, 18–21.

Silberman, M. S.; Irani, L.; and Ross, J. 2010. Ethics and tactics of professional crowdwork. XRDS 17(2):39–43.

Singer, Y., and Mittal, M. 2011. Pricing mechanisms for online labor markets. In Third AAAI Human Computation Workshop (HCOMP’11).

Full Text: PDF