(631c) Stochastic Model Predictive Control with Deep Generative Disturbance Models: Application to Control of Building Energy Systems

Model predictive control (MPC) is an advanced control technology that optimizes the sequence of control actions based on predictions of future cost and constraints using an internal system model [1, 2]. Only the first control action is implemented, after which new system measurements are collected and integrated into the prediction model, and then the entire optimization-based planning process is repeated given this new data. As such, MPC is a flexible control paradigm that can handle multiple competing control objectives, constraints on the control actions and desired outcomes, and different types of system measurements. By formulating and solving an optimal control problem (a multi-step optimization-based planning problem) at every iteration, MPC can anticipate challenges that may arise due to the specific form of the dynamics and/or constraints and thus can preemptively act before these challenges occur. Further, by continually re-solving the optimal control problem, MPC can detect and adjust to perturbations yielding a form of inherent robustness due to feedback from measurements. However, the optimal control actions suggested by MPC strongly depend on the quality of the system model and exogenous disturbance forecast. In most cases, the disturbances are realized from some underlying probability distribution and so selecting a single point estimate (as done in nominal MPC) can result in overly aggressive control actions. Stochastic MPC (SMPC) is an extension of standard MPC that looks to consider the full distribution of uncertainty at every iteration [3]. Although conceptually simple, a major technical challenge in SMPC is the propagation of uncertainty through the system dynamics to accurately predict the (joint) probability distribution of states and actions over time. A variety of different uncertainty propagation schemes have been proposed to deal with this challenge such as scenario trees [4], moment-based approximations [5], and polynomial chaos expansions [6]. However, these works typically make strong assumptions about the underlying stochastic process for the disturbances such as their realizations are independent at every sample time. There has been limited work on more general disturbance processes that exhibit complex correlation structures over time.

Probabilistic (deep) neural networks offer a scalable and automated framework for learning complex conditional distributions directly from data. Two common approaches include the use of generative networks, such as conditional variational autoencoders (CVAEs) [7], that construct disturbance signals over a fixed time horizon, or regressive networks that take a previous time window of disturbances as inputs and predict the disturbances over a future time window. While both classes of models have been demonstrated to be useful, e.g., [7, 8], we have empirically found that generative models perform best in data-limited settings. While generative models have been used in the closed-loop performance verification of control policies [9], there has been little-to-no work on leveraging generative models for SMPC design. In this work, we propose τ-SMPC, a Trajectory Adapting Uncertainty (TAU=τ) framework that facilitates scenario-based SMPC to leverage samples from complex generative disturbance models. The key novelty in our approach is a method for sequentially sampling from the latent space of a deep generative model (such as a CVAE) to produce realistic out-of-sample disturbance scenarios given partially revealed disturbance trajectories. Through simulation experiments on a building energy management system, we demonstrate τ-SMPC’s ability to simultaneously reduce operational costs and satisfy temperature constraints compared to alternative methods.

References:

[1] Mayne, D. Q. (2014). Model predictive control: Recent developments and future promise. Automatica, 50(12), 2967-2986.

[2] Rawlings, James Blake, David Q. Mayne, and Moritz Diehl. Model predictive control: theory, computation, and design. Vol. 2. Madison, WI: Nob Hill Publishing, 2017.

[3] Mesbah, A. (2016). Stochastic model predictive control: An overview and perspectives for future research. IEEE Control Systems Magazine, 36(6), 30-44.

[4] Lucia, Sergio, Tiago Finkler, and Sebastian Engell. "Multi-stage nonlinear model predictive control applied to a semi-batch polymerization reactor under uncertainty." Journal of process control 23.9 (2013): 1306-1319.

[5] Paulson, J. A., Mesbah, A., Streif, S., Findeisen, R., & Braatz, R. D. (2014, December). Fast stochastic model predictive control of high-dimensional systems. In 53rd IEEE Conference on decision and Control (pp. 2802-2809). IEEE.

[6] Mesbah, A., Streif, S., Findeisen, R., & Braatz, R. D. (2014, June). Stochastic nonlinear model predictive control with probabilistic constraints. In 2014 American control conference (pp. 2413-2419). IEEE.

[7] Kumar, P., Rawlings, J. B., Wenzel, M. J., & Risbeck, M. J. (2023). Grey-box model and neural network disturbance predictor identification for economic MPC in building energy systems. Energy and Buildings, 286, 112936.

[8] Salatiello, Alessandro, Ye Wang, Gordon Wichern, Toshiaki Koike-Akino, Yoshihiro Ohta, Yosuke Kaneko, Christopher Laughman, and Ankush Chakrabarty. "Synthesizing Building Operation Data with Generative Models: VAEs, GANs, or Something In Between?." In Companion Proceedings of the 14th ACM International Conference on Future Energy Systems, pp. 125-133. 2023.

[9] Khayatian, F., Nagy, Z., & Bollinger, A. (2021). Using generative adversarial networks to evaluate robustness of reinforcement learning agents against uncertainties. Energy and Buildings, 251, 111334.