(109b) Integrating Deep Reinforcement Learning and Model Predictive Control: A Practical Framework for Industrial Applications

In this presentation, we introduce a framework of Deep Reinforcement Learning Based Model Predictive Control (DRLMPC), which combines Deep Reinforcement Learning (DRL) and Model Predictive Control (MPC) to overcome the limitations of each method when used independently. MPC, while popular in industry, often falls short of achieving optimal results due to the finite prediction horizons dictated by computational constraints and uncertainties in the model. Conversely, Reinforcement Learning excels in long-term decision making in a stochastic environment but often requires extensive, and potentially hazardous, online data collection for training.

Our proposed framework tackles these challenges by integrating a neural network-based value function approximator as a terminal cost within the MPC structure. This value function, central to reinforcement learning, is continually refined online using operational data from closed-loop systems, akin to policy iteration in dynamic programming. This iterative enhancement fosters evolutionary performance gains and adaptability to shifts in long-term dynamics. Consequently, DRLMPC significantly extends the prediction horizon, robustly counters stochastic disturbances, and reduces dependence on extensive online data collection via trial and error, thus boosting both safety and operational efficiency.

To demonstrate the practical effectiveness of DRLMPC, we implemented the algorithm in a common Continuous Stirred Tank Reactor (CSTR) control scenario. Our evaluation indicates that DRLMPC proficiently resolves the typically neglected terminal cost issue in conventional MPC frameworks. It also skillfully accommodates model-plant discrepancies and mitigates the effects of unmeasured stochastic disturbances within a computationally feasible prediction horizon. These findings highlight DRLMPC as a formidable control strategy that integrates the structured precision of MPC with the adaptive learning capabilities of deep reinforcement learning, leading to substantial improvements in the performance of complex industrial control systems.