AAAI Publications, Twenty-Ninth AAAI Conference on Artificial Intelligence

Font Size: 
Integration and Evaluation of a Matrix Factorization Sequencer in Large Commercial ITS
Carlotta Schatten, Ruth Janning, Lars Schmidt-Thieme

Last modified: 2015-02-16

Abstract


Correct evaluation of Machine Learning based sequencers require large data availability, large scale experiments and consideration of different evaluation measures. Such constraints make the construction of ad-hoc Intelligent Tutoring Systems (ITS) unfeasible and impose early integration in already existing ITS, which possesses a large amount of tasks to be sequenced. However, such systems were not designed to be combined with Machine Learning methods and require several adjustments. As a consequence more than a half of the components based on recommender technology are never evaluated with an online experiment. In this paper we show how we adapted a Matrix Factorization based performance predictor and a score based policy for task sequencing to be integrated in a commercial ITS with over 2000 tasks on 20 topics. We evaluated the experiment under different perspectives in comparison with the ITS sequencer designed by experts over the years. As a result we achieve same post-test results and outperform the current sequencer in the perceived experience questionnaire with almost no curriculum authoring effort. We also showed that the sequencer possess a better user modeling, better adapting to the knowledge acquisition rate of the students.

Keywords


Machine Learning;Sequencing; Matrix Factorization; Vygotsky;ITS;

Full Text: PDF