Image copyrights basti_m28

Welcome to RLEM 2021

About RLEM'21

RLEM brings together researchers and industry practitioners for the advancement of (deep) reinforcement learning (RL) in the built environment as it is applied for managing energy in civil infrastructure systems (energy, water, transportation). Following BuildSys's directive, the conference will be held virtually in November 17th 2021. More information about how to join the virtual sessions will be posted here soon.

RLEM'21 will be held in conjunction with ACM BuildSys'21

Important Dates

Abstract submission

September 6, 2021 (AOE)

Paper submission

September 6, 2021 (AOE)


September 27, 2021 (AOE)

Camera Ready

October 1, 2021 (AOE)

Workshop date

November 16, 2021

Call for Papers

Buildings account for 40% of the global energy consumption and 30% of the associated greenhouse gas emissions, while also offering a 50–90% CO2 mitigation potential. The transportation sector is responsible for an additional 30%. Optimal decarbonization requires electrification of end-uses and concomitant decarbonization of electricity supply, efficient use of electricity for lighting, space heating, cooling and ventilation (HVAC), and domestic hot water generation, and upgrade of the thermal properties of buildings. A major driver for decarbonization are integration of renewable energy systems (RES) into the grid, and photovoltaics (PV) and solar-thermal collectors as well as thermal and electric storage into residential and commercial buildings. Electric vehicles (EVs), with their storage capacity and inherent connectivity, hold a great potential for integration with buildings.

The integration of these technologies must be done carefully to unlock their full potential. Artificial intelligence is regarded as a possible pathway to orchestrate these complexities of Smart Cities. In particular, (deep) reinforcement learning algorithms have seen an increased interest and have demonstrated human expert level performance in other domains, e.g., computer games. Research in the building and cities domain has been fragmented and with focus on different problems and using a variety of frameworks. The purpose of this Workshop is to build a growing community around this exciting topic, provide a platform for discussion for future research direction, and share common frameworks.

Topics of Interest

Topics of interest include, but are not limited to:

  • Challenges and Opportunities for RL in Building and Cities
  • Explorations of model vs model-free RL algorithms and hybrids
  • Comparisons of RL algorithms to other control solutions, e.g., model-predictive control
  • Frameworks and datasets for benchmarking algorithms
  • Theoretical contributions to the RL field brought about by constraints/challenges in the buildings/cities domain
  • Applications (demand response, HVAC control, occupant integration, traffic scheduling, EV/battery charging, DER integration)

Submission Instructions

Submitted papers must be unpublished and must not be currently under review for any other publication. Paper submissions must be at most 4 single-spaced US Letter (8.5"x11") pages, including figures, tables, and appendices (excluding references). All submissions must use the LaTeX (preferred) or Word styles found here Authors must make a good faith effort to anonymize their submissions by (1) using the "anonymous" option for the class and (2) using "anonsuppress" section where appropriate. Papers that do not meet the size, formatting, and anonymization requirements will not be reviewed. Please note that ACM uses 9-pt fonts in all conference proceedings, and the style (both LaTeX and Word) implicitly define the font size to be 9-pt.

Submission Link


Registration: Register now via BuildSys 2021


All times in Western European timezone (GMT+1)

Time Presentation
15:00 - 15:20 Opening remarks, by General Chair and TPC Chairs
15:20 - 16:00 1st Keynote: Andrey Bernstein (National Renewable Energy Laboratory)
Session 1: Addressing challenges of applying RL in real-world buildings
16:00 - 16:50 A. Naug (Vanderbilt University), M. Quinones-Grueiro (Vanderbilt University), G. Biswas (Vanderbilt University) - Sensitivity and Robustness of End-to-end Data-Driven Approach for Building Performance Op-timization
M. Biemann (TU Denmark), X. Liu (TU Denmark), Y. Zeng (Northumbria University), L. Huang (Norwegian University of Science and Technology) - Addressing partial observability in reinforcement learning for energy management
K. Jneid (Université Grenoble Alpes, LIG), S. Ploix (Grenoble INP), P. Reignier (Université Greno-ble Alpes, LIG), P. Jallon (eLichens) - Deep Q-Network Boosted with External Knowledge for HVAC Control
16:50 - 17:00 Break
17:00 - 17:40 2nd Keynote: Helia Zandi (Oak Ridge National Laboratory)
17:40 - 18:00 Community Announcements (CityLearn Challenge Winners, Annex#81, Climate Change AI)
Session 2: Benchmarking RL with other controls including other RLs
18:00 - 18:50 K. Kurte (ORNL), K. Amasyali (ORNL), J. Munk(ORNL), H. Zandi (ORNL) - Comparative Analysis of Model-Free and Model-Based HVAC Control for Residential Demand Response
J. Jiménez-Raboso (Universidad de Granada), A. Campoy-Nieves (Universidad de Granada), A. Manjavacas-Lucas (Universidad de Granada), J. Gómez-Romero (Universidad de Granada), M. Molina-Solana (Universidadde Granada) - Sinergym: A Building Simulation and Control Framework for Training Reinforcement Learning Agents
R. Glatt (LLNL), F. Leno da Silva (LLNL), B. Soper (LLNL), W. A. Dawson (LLNL), E. Rusu (LLNL), R. A. Goldhahn (LLNL) - Collaborative energy demand response with decentralized actor and centralized critic
18:50 - 19:00 Break
Session 3: Testing environment
19:00 - 20:20 COBS: COmprehensive Building Simulator
BOPTEST: Building Optimization Performance Test framework
ACTB: Advanced Controls Test Bed
20:20 - 20:30 Closing remarks

Keynote Speaker

Helia Zandi (Modeling and Simulation Software Engineer at Oak Ridge National Laboratory)


Abstract: TBD

Bio: TBD

Andrey Bernstein (Group Manager at National Renewable Energy Laboratory)


Abstract: TBD

Bio: TBD


General Chair

  • Zoltan Nagy (University of Texas at Austin)

Technical Program Committee Co-Chairs

Web Chair

Technical Program Committee