AAAI Publications, Workshops at the Twenty-Sixth AAAI Conference on Artificial Intelligence

Font Size: 
Predicting Crowd-Based Translation Quality with Language-Independent Feature Vectors
Nina Runge, Niklas Kilian, Jan Smeddinck, Markus Krause

Last modified: 2012-07-15


Research over the past years has shown that machine translation results can be greatly enhanced with the help of mono- or bilingual human contributors, e.g. by asking hu- mans to proofread or correct outputs of machine translation systems. However, it remains difficult to determine the quality of individual revisions. This paper proposes a meth- od to determine the quality of individual contributions by analyzing task-independent data. Examples of such data are completion time, number of keystrokes, etc. An initial evaluation showed promising F-measure values larger than 0.8 for support vector machine and decision tree based classifications of a combined test set of Vietnamese and German translations.


human computation; crowdsourcing; machine translation; answer prediction; rater reliability analysis

Full Text: PDF