Toward a Comprehension Challenge, Using Crowdsourcing as a Tool

Praveen Paritosh, Gary Marcus

Abstract


Human readers comprehend vastly more, and in vastly different ways, than any existing comprehension test would suggest. An ideal comprehension test for a story should cover the full range of questions and answers that humans would expect other humans to reasonably learn or infer from a given story. As a step toward these goals we propose a novel test, the Crowdsourced Comprehension Challenge (C3), which is constructed by repeated runs of a three-person game, the Iterative Crowdsourced Comprehension Game (ICCG). ICCG uses structured crowdsourcing to comprehensively generate relevant questions and supported answers for arbitrary stories, whether fiction or nonfiction, presented across a variety of media such as videos, podcasts, and still images.

Full Text:

PDF


DOI: https://doi.org/10.1609/aimag.v37i1.2649

Copyright © 2017, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved.