AAAI Publications, Twenty-Seventh AAAI Conference on Artificial Intelligence

Font Size: 
Online Optimization with Dynamic Temporal Uncertainty: Incorporating Short Term Predictions for Renewable Integration in Intelligent Energy Systems
Vikas Garg, T. S. Jayram, Balakrishnan Narayanaswamy

Last modified: 2013-06-29


Growing costs, environmental awareness and government directives have set the stage for an increase in the fraction of electricity supplied using intermittent renewable sources such as solar and wind energy. To compensate for the increased variability in supply and demand, we need algorithms for online energy resource allocation under temporal uncertainty of future consumption and availability. Recent advances in prediction algorithms offer hope that a reduction in future uncertainty, through short term predictions, will increase the worth of the renewables. Predictive information is then revealed incrementally in an online manner, leading to what we call dynamic temporal uncertainty. We demonstrate the non-triviality of this problem and provide online algorithms, both randomized and deterministic, to handle time varying uncertainty in future rewards for non-stationary MDPs in general and for energy resource allocation in particular. We derive theoretical upper and lower bounds that hold even for a finite horizon, and establish that, in the deterministic case, discounting future rewards can be used as a strategy to maximize the total (undiscounted) reward. We also corroborate the efficacy of our methodology using wind and demand traces.


Reinforcement Learning; Markov Decision Process (MDP); Robust Optimization; Smart grid; Storage management; Online learning; Optimization; Control systems; Imprecise MDPs; Energy; Uncertainty; Decision making; Non-stationary policies; MPC; Regret

Full Text: PDF