AAAI Publications, Second AAAI Conference on Human Computation and Crowdsourcing

Font Size: 
Robot Programming by Demonstration with Crowdsourced Action Fixes
Maxwell Forbes, Michael Jae-Yoon Chung, Maya Cakmak, Rajesh P. N. Rao

Last modified: 2014-09-05


Programming by Demonstration (PbD) can allow end-users to teach robots new actions simply by demonstrating them. However, learning generalizable actions requires a large number of demonstrations that is unreasonable to expect from end-users. In this paper, we explore the idea of using crowdsourcing to collect action demonstrations from the crowd. We propose a PbD framework in which the end-user provides an initial seed demonstration, and then the robot searches for scenarios in which the action will not work and requests the crowd to fix the action for these scenarios. We use instance-based learning with a simple yet powerful action representation that allows an intuitive visualization of the action. Crowd workers directly interact with these visualizations to fix them. We demonstrate the utility of our approach with a user study involving local crowd workers (N=31) and analyze the collected data and the impact of alternative design parameters so as to inform a real-world deployment of our system.


Robotics; Crowdsourcing; Programming by Demonstration; Active Learning


Abbeel, P.; Coates, A.; and Ng, A. 2010. Autonomous he- licopter aerobatics through apprenticeship learning. The In- ternational Journal of Robotics Research.

Aha, D. W.; Kibler, D.; and Albert, M. K. 1991. Instance- based learning algorithms. Machine learning 6(1):37–66.

Akgun, B.; Cakmak, M.; Wook Yoo, J.; and Thomaz, L. A. 2012. Trajectories and keyframes for kinesthetic teaching: A human-robot interaction perspective. In Proceedings of the ACM/IEEE International Conference on Human-Robot Interaction (HRI).

Alexandrova, S.; Cakmak, M.; Hsiao, K.; and Takayama, L. 2014. Robot programming by demonstration with interac- tive action visualizations. In Robotics: science and systems, 48–56. Berkeley, CA.

Argall, B.; Chernova, S.; Veloso, M. M.; and Browning, B. 2009. A survey of robot learning from demonstration. Robotics and Autonomous Systems 57(5):469–483.

Atkeson, C. G.; Moore, A. W.; and Schaal, S. 1997. Locally weighted learning for control. In Lazy learning. Springer. 75–113.

Billard, A.; Calinon, S.; Dillmann, R.; and Schaal, S. 2008. Robot Programming by Demonstration. In Handbook of Robotics. chapter 59.

Breazeal, C., and Thomaz, A. L. 2008. Learning from hu- man teachers with socially guided exploration. In Proceed- ings of the IEEE International Conference on Robotics and Automation (ICRA).

Breazeal, C.; DePalma, N.; Orkin, J.; Chernova, S.; and Jung, M. 2013. Crowdsourcing human-robot interaction: New methods and system evaluation in a public environ- ment. Journal of Human-Robot Interaction 2(1):82–111.

Cakmak, M.; Chao, C.; and Thomaz, A. L. 2010. Designing interactions for robot active learners. Autonomous Mental Development, IEEE Transactions on 2(2):108–118.

Calinon, S., and Billard, A. 2009. Statistical learning by imitation of competing constraints in joint and task space. Advanced Robotics 23(15):2059–2076.

Chernova, S., and Veloso, M. 2009. Interactive policy learn- ing through confidence-based autonomy. Journal of Artifi- cial Intelligence Research 34.

Chernova, S.; DePalma, N.; Morant, E.; and Breazeal, C. 2011. Crowdsourcing human-robot interaction: Application from virtual to physical worlds. In RO-MAN, 2011 IEEE, 21–26. IEEE.

Chung, M.; Forbes, M.; Cakmak, M.; and Rao, R. 2014. Ac- celerating imitation learning through crowdsourcing. In Pro- ceedings of the IEEE international Conference on Robotics and Automation (ICRA).

Crick, C.; Osentoski, S.; Jay, G.; and Jenkins, O. C. 2011. Human and robot perception in large-scale learning from demonstration. In Proceedings of the 6th international con- ference on Human-robot interaction, 339–346. ACM.

Emeli, V. 2012. Robot learning through social media crowd- sourcing. In Intelligent Robots and Systems (IROS), 2012 IEEE/RSJ International Conference on, 2332–2337.

Koenig, N.; Takayama, L.; and Mataric ́, M. 2010. Commu- nication and knowledge sharing in human–robot interaction and learning from demonstration. Neural Networks 23(8).

Kormushev, P.; Calinon, S.; and Caldwell, D. G. 2010. Robot motor skill coordination with EM-based reinforce- ment learning. In Proc. IEEE/RSJ Intl Conf. on Intelligent Robots and Systems (IROS), 3232–3237.

Muelling, K.; Kober, J.; Kroemer, O.; and Peters, J. 2013. Learning to select and generalize striking movements in robot table tennis. International Journal of Robotics Re- search (3):263–279.

Pastor, P.; Hoffmann, H.; Asfour, T.; and Schaal, S. 2009. Learning and generalization of motor skills by learning from demonstration. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA).

Saunders, J.; Otero, N.; and Nehaniv, C. 2007. Issues in human/robot task structuring and teaching. In Proceedings of the IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), 708 – 713.

Schaal, S.; Peters, J.; Nakanishi, J.; and Ijspeert., A. 2003. Learning movement primitives. In Proceedings of the Inter- national Symposium on Robotics Research (ISRR).

Schulman, J.; Ho, J.; Lee, C.; and Abbeel, P. 2013. General- ization in robotic manipulation through the use of non-rigid registration. Proceedings of the 16th International Sympo- sium on Robotics Research (ISRR).

Settles, B. 2012. Active learning literature survey. Computer Sciences Technical Report 1648, University of Wisconsin– Madison.

Sorokin, A.; Berenson, D.; Srinivasa, S. S.; and Hebert, M. 2010. People helping robots helping people: Crowdsourcing for grasping novel objects. In Intelligent Robots and Systems (IROS), 2010 IEEE/RSJ International Conference on, 2117– 2122. IEEE.

Suay, H. B.; Toris, R.; and Chernova, S. 2012a. A practical comparison of three robot learning from demonstration al- gorithm. International Journal of Social Robotics 4(4):319– 330.

Suay, H.; Toris, R.; and Chernova, S. 2012b. A practical comparison of three robot learning from demonstration al- gorithms. Intl. Journal of Social Robotics, special issue on LfD 4(4).

Toris, R.; Kent, D.; and Chernova, S. 2014. The robot management system: A framework for conducting human- robot interaction studies through crowdsourcing. Journal of Human-Robot Interaction.

Verma, D., and Rao, R. P. 2006. Goal-based imitation as probabilistic inference over graphical models. Advances in neural information processing systems 18:1393.

Full Text: PDF