PhD Proposal: Daniel Urieli, GDC 3.516

Contact Name: 
Lydia Griffith
Dec 4, 2013 1:00pm - 3:00pm

PhD Proposal: Daniel Urieli

Date: Dec. 4th, 2013
Time: 1 pm
Place: GDC 3.516
Supervisor: Prof. Peter Stone

Title: Learning Agents for Sustainable Energy


To ensure sustainable existence for society, future energy
generation will have to use renewable resources rather than
fossil-based fuels.  The reduction in usage of fossil fuels is
predicted to increase electricity demand significantly.  Satisfying
this increased demand using renewable energy resources creates a
variety of new challenges that require a major change in the current
electricity infrastructure.  To address these challenges,
governments are re-engineering their electricity infrastructure into
a \emph{smart-grid} which, among other things, is expected to
include a significant component of artificial intelligence (AI)
technologies. Specifically, autonomous agents controlling different
components of the smart grid are expected to make robust real-time
decisions in fast paced, information rich, complex environments
which are frequently initially-unknown, sequential, dynamic,
stochastic,  partially-observable, continuous, high-dimensional, and
possibly non-stationary.  These complex environments create new
challenges for AI as a research field.

To cope with these challenges, this thesis takes an
application-motivated approach, by focusing on two realistic
problems of practical interest, and solving them using novel
algorithmic contributions. Specifically, these motivating problems
are: (1) HVAC thermostat control for minimizing energy consumption
while satisfying comfort requirements, and (2) energy trading in a
smart-grid environment, with financial incentives that encourage
sustainable behaviors.  Motivated by these application domains, this
thesis makes 4 main contributions.  First, this thesis formalizes
these problems as Reinforcement Learning (RL) problems and
characterizes their optimal solutions. Since these problems are
intractable, one must resort to approximate solutions. Second, this
thesis contributes novel online RL algorithms developed for these
problems. Third, this thesis contributes complete, fully implemented
agents that are competitive with or exceed the state-of the art
solutions to these problems, and which integrate the above RL
algorithms into complete planning architectures.  Fourth, this
thesis contributes detailed empirical evaluations, of the agents'
components, the complete agents, and their impacts on the
smart-grid.  Taken together, the contributions of this thesis are
expected to help lead the way towards achieving the vision of a
smart grid, while at the same time contributing to the vibrant
scientific community focusing on the development of robust, broadly
applicable RL algorithms.