(177c) Learning Based Scheduling of Industrial Hybrid Renewable Energy Systems

Due to the high intermittent nature of renewable energy sources, the system complexity increases, thereby making the optimal decision making more challenging using the existing physics-based model techniques. The inevitable uncertainties related to energy sector such as climate changes, energy market fluctuations, and energy technology developments often demand complex physics-based model and excessive computational time especially for control applications. The physics-based energy scheduling approach generally solves an optimization problem in background to decide the optimal power mix among the available power sources. However, model dependencies, parameter uncertainties and multi-dimensional impact of the embedded uncertainties impose the decision making to be more problematic, especially when such an approach is extended to large-scale and complex energy systems.

The utilization of machine learning techniques in the energy sector has gradually attracted researchers’ attention for the effective management of the aforementioned complexities due to their purely data-driven nature. Reinforcement learning (RL), which is a branch of machine learning, can adopt model-free approaches for scheduling under uncertainties by employing agents to participate in the process of interest. The RL agents make control decisions and maximize its total reward by interacting with the simulation environment and evaluating the series of actions taken.

In this study, a data-driven approach that adopts RL techniques to determine the schedule of power mix in a hybrid renewable energy system (HRES) is presented, with a case study on the highly energy intensive chlor-alkali process. This process involves electrolysis of sodium chloride solution to produce chlorine as the primary product together with hydrogen and sodium hydroxide (caustic soda) solution as the secondary products. Almost 50-70% of the overall cost of operating a chlor-alkali process comes from electricity usage, thereby making the process highly sensitive to electricity price trends. To combat the increased grid dependency of such a highly energy intensive process and to achieve profitability, a grid connected hybrid energy system comprising of solar PV panels, wind turbines and fuel cells are considered that supplies power to the chlor-alkali process.

We implement Proximal Policy Optimization (PPO), an Advantage Actor-Critic RL method that enable samples to be trained for multiple epochs of mini-batch updates, to effectively distribute energy among the various available sources without having access to the dynamics of the system. In our exclusive experiments, the RL based scheduling provides efficient operational strategies with saving in cost and show nearly on-par performance with the optimal solution. For stochastic tasks under uncertainties, our RL scheduling approach demonstrates strong superiority over the conventional optimization. This framework can generally be used in any data-driven scheduling problems taking the advantages of its model-free property, and can be generalized to multiple task scenarios via transfer learning.