Second ACM SIGEnergy Workshop on Reinforcement Learning for Energy Management in Buildings & Cities (RLEM)

Keynote Speaker

Helia Zandi (Modeling and Simulation Software Engineer at Oak Ridge National Laboratory)

Scalable Load Management System using Reinforcement Learning

Abstract: Internet of things has revolutionized the interaction between devices, actuators, sensors, and individuals in a building. Effective integration of these assets in buildings has critical role in providing load flexibility to support grid resiliency. In addition to a frequently changing load shape due to new demand patterns, the increasing penetration of renewable energy sources adds variability to power generation. The grid needs a reliable control system to coordinate the effects of these changes in real time to ensure safe and reliable operation. To address this, many optimization methods have been developed and researched. However, many of the existing approaches cannot adapt to the changes in the environment. Model-free reinforcement learning has gained a lot of attention in recent years for creating the optimal load schedule in buildings to provide benefits for building and utility stakeholders. In this talk, we will discuss a scalable software framework and various deep model-free RL algorithms that we have developed, and field tested over the past few years for optimal operation of the buildings.

Bio: Helia Zandi received her M.S. in Computer Engineering from University of Florida in 2012. She is currently a Reseacrh and Development Staff in the Computational Systems Engineering Group in Computational Sciences and Engineering Division at Oak Ridge National Laboratory (ORNL). She joined ORNL as a Modeling and Simulation Software Engineer in 2016. Prior to that, she worked at ProNova Solutions as a Researcher and Software Engineer in Imaging and Positioning team. She developed sophisticated automation methodologies for accurate robot positioning and robot calibration. Her research expertise includes machine learning, robotics, advance data analytics, cyber-physical system, large-scale building-to-grid integration, and design and developing optimization and control platform with an application to buildings and energy systems.

Andrey Bernstein (Group Manager at National Renewable Energy Laboratory)

Data-driven Control of Complex Engineering Systems

Abstract: Optimal control of complex engineering systems is an extremely hard task. The classical approach, based on optimal control concepts (such as model-predictive control), is infeasible for large-scale systems where the accurate model is unavailable or is expensive to develop. Thus, machine learning (ML) approaches are becoming a popular alternative. In this talk, we will overview two most popular ML-based approaches, one based on reinforcement learning and another based on data-driven predictive control, their variants, and their application to optimal control of grid-interactive buildings.

Bio: Andrey Bernstein received his B.Sc., M.Sc., and PhD degrees in Electrical Engineering from the Technion - Israel Institute of Technology. Between 2010 and 2011, he was a visiting researcher at Columbia University. During 2011-2012, he was a visiting Assistant Professor at the Stony Brook University. From 2013 to 2016, he was a postdoctoral researcher at the Laboratory for Communications and Applications of Ecole Polytechnique Federale de Lausanne (EPFL), Switzerland. Since October 2016 he has been a Senior Researcher and Group Manager at the National Renewable Energy Laboratory, Golden, CO, USA. His research interests are in the decision and control problems in complex environments and related optimization and machine learning methods, with application to power and energy systems.