AAAI Publications, Twenty-Sixth AAAI Conference on Artificial Intelligence

Font Size: 
An Intelligent Battery Controller Using Bias-Corrected Q-learning
Donghun Lee, Warren B Powell

Last modified: 2012-07-12


The transition to renewables requires storage to help smooth short-term variations in energy from wind and solar sources, as well as to respond to spikes in electricity spot prices, which can easily exceed 20 times their average. Efficient operation of an energy storage device is a fundamental problem, yet classical algorithms such as $Q$-learning can diverge for millions of iterations, limiting practical applications. We have traced this behavior to the max-operator bias, which is exacerbated by high volatility in the reward function, and high discount factors due to the small time steps. We propose an elegant bias correction procedure and demonstrate its effectiveness.


Computational Sustainability; Energy; Q Learning; Max-induced Bias; Battery Control;

Full Text: PDF