AAAI Publications, Twenty-Ninth AAAI Conference on Artificial Intelligence

Font Size: 
High-Confidence Off-Policy Evaluation
Philip S. Thomas, Georgios Theocharous, Mohammad Ghavamzadeh

Last modified: 2015-02-21


Many reinforcement learning algorithms use trajectories collected from the execution of one or more policies to propose a new policy. Because execution of a bad policy can be costly or dangerous, techniques for evaluating the performance of the new policy without requiring its execution have been of recent interest in industry. Such off-policy evaluation methods, which estimate the performance of a policy using trajectories collected from the execution of other policies, heretofore have not provided confidences regarding the accuracy of their estimates. In this paper we propose an off-policy method for computing a lower confidence bound on the expected return of a policy.


policy evaluation; high-confidence; concentration inequality

Full Text: PDF