AAAI Publications, Second AAAI Conference on Human Computation and Crowdsourcing

Font Size: 
Crowdsourcing in Language Classes Can Help Natural Language Processing
Barbora Hladká, Jirka Hana, Ivana Lukšová

Last modified: 2014-09-05


One way of teaching grammar, namely morphology and syntax, is to visualize sentences as diagrams capturing relationships between words. Similarly, such relationships are captured in a more complex way in treebanks serving as key building stones in modern natural language processing. However, building them is very time consuming, thus we have been seeking for an alternative cheaper and faster way, like crowdsourcing. The purpose of our work is to explore possibility to get sentence diagrams produced by students and teachers. In our pilot study, the object language is Czech, where sentence diagrams are part of elementary school curriculum.


crowdsourcing, education, syntax, morphology,


Bille, P. 2005. A survey on tree edit distance and related problems. Theoretical computer science 337(1):217–239. Brabham, D. C. 2013. Crowdsourcing. MIT Press.

Hana, J., and Hladka, B. 2012. Getting more data: Schoolkids as annotators. In Proceedings of the 8th International Conference on Language Resources and Evaluation (LREC 2012).

Hana, J.; Hladka, B.; and Luksova, I. 2014. Sentence diagrams: their evaluation and combination. In Proceedings of the 8th Linguistic Annotation Workshop (LAW VIII 2014).

Hsueh, P.-Y.; Melville, P.; and Sindhwani, V. Data quality from crowdsourcing: A study of annotation selection criteria. In Proceedings of the NAACL HLT 2009 Workshop on Active Learning for Natural Language Processing.

Petrov, S., and McDonald, R. 2012. Overview of the 2012 Shared Task on Parsing the Web. Notes of the First Workshop on Syntactic Analysis of Non-Canonical Language (SANCL).

Full Text: PDF