AAAI Publications, Twenty-Eighth AAAI Conference on Artificial Intelligence

Font Size: 
Imitation Learning with Demonstrations and Shaping Rewards
Kshitij Judah, Alan Paul Fern, Prasad Tadepalli, Robby Goetschalckx

Last modified: 2014-06-21


Imitation Learning (IL) is a popular approach for teaching behavior policies to agents by demonstrating the desired target policy. While the approach has lead to many successes, IL often requires a large set of demonstrations to achieve robust learning, which can be expensive for the teacher. In this paper, we consider a novel approach to improve the learning efficiency of IL by providing a shaping reward function in addition to the usual demonstrations. Shaping rewards are numeric functions of states (and possibly actions) that are generally easily specified, and capture general principles of desired behavior, without necessarily completely specifying the behavior. Shaping rewards have been used extensively in reinforcement learning, but have been seldom considered for IL, though they are often easy to specify. Our main contribution is to propose an IL approach that learns from both shaping rewards and demonstrations. We demonstrate the effectiveness of the approach across several IL problems, even when the shaping reward is not fully consistent with the demonstrations.


Sequential Decision Making; Imitation Learning; Reinforcement Learning; Reward Shaping

Full Text: PDF