Industrial processes are subject to disturbances and sometimes inherent stochastic fluctuations. Disturbances may arise from a multitude of different causes and introduce uncertainty in the form of noise into the system. This noise is typically considered white noise in process models for control applications such as stochastic model predictive control (SMPC) (Rawlings et al., 2017; Mesbah, 2016). However, there are cases where the stochastic fluctuations (nonlinearly) depend on the system states, which is typically not considered in SMPC models, e.g., reaction kinetics in biological systems with random mutations depending on the concentrations (Álvarez et al., 2018). In such cases, SMPC models should account for the state dependency of the fluctuations.
This work uses conditional normalizing flows (Papamakarios et al., 2021) to design probabilistic state space models for SMPC. Normalizing flows are flexible, data-driven probability distribution models for high-dimensional data that allow for density estimation and scenario generation (Papamakarios et al., 2021). Previous works by the author have extended baseline normalizing flows to conditional distributions to form a forecasting method for renewable electricity generation (Cramer et al., 2022, 2023). However, the concept generalizes to any other type of probabilistic regression task.
In a discrete-time setting, this work uses the conditional normalizing flow to learn the conditional joint probability distribution p(x[k+1]∣x[k],u[k]) of the system state vector in the next time step x[k+1] based on the current states x[k] and control inputs u[k]. Notably, normalizing flows makes no assumptions about the probability distribution or the conditional dependence. Hence, the conditional normalizing flow presents a fully flexible probabilistic state space model for nonlinear systems with state-dependent and even non-Gaussian noise. Furthermore, normalizing flows provide an explicit expression for the log-likelihood functions of the joint probability distribution of the following time step. This log-likelihood function allows the user to formulate a probabilistic state-tracking objective that maximizes the likelihood of achieving the setpoints instead of minimizing the expected mean squared error.
The performance of the normalizing flow state space model is evaluated in case studies of the autonomous Lotka-Volterra system and SMPC of a continuous stirred tank reactor (CSTR) (Bequette, 1998). Both systems are simulated by solving nonlinear stochastic differential equations (SDE) using the Euler-Maruyama method (Thygesen, 2023). The normalizing flows are implemented in TensorFlow (Abadi and Agarwal, 2015), and the SMPC optimization is solved via the automatic differentiation wrapper in autograd-minimize (Rigal, 2023) and the Python-based optimization library scipy-optimize (Virtanen et al., 2020). The normalizing flow state space model accurately learns the stochastic dynamics of the systems and yields stable predictions over many time steps. The SMPC formulations give good state-tracking performance for both open and closed-loop settings.
Abadi, M., & Agarwal, A. (2015). TensorFlow: Large-scale machine learning on heterogeneous systems. Retrieved from https://www.tensorflow.org/. Accessed on 08-08-2022.
Bequette, B. W. (1998). Process dynamics: Modeling, analysis, and simulation. Lecture notes.
Cramer, E., Paeleke, L., Mitsos, A., & Dahmen, M. (2022). Normalizing flow-based day-ahead wind power scenario generation for profitable and reliable delivery commitments by wind farm operators. Computers & Chemical Engineering, 166, 107923.
Cramer, E., Witthaut, D., Mitsos, A., & Dahmen, M. (2023). Multivariate probabilistic forecasting of intraday electricity prices using normalizing flows. Applied Energy, 346, 121370.
Mesbah, A. (2016). Stochastic model predictive control: An overview and perspectives for future research. IEEE Control Systems, 36(6), 30–44.
Papamakarios, G., Nalisnick, E., Rezende, D. J., Mohamed, S., & Lakshminarayanan, B. (2021). Normalizing flows for probabilistic modeling and inference. Journal of Machine Learning Research, 22(57), 1–64.
Rawlings, J. B., Mayne, D. Q., Diehl, M., et al. (2017). Model Predictive Control: Theory, Computation, and Design (Vol. 2). Nob Hill Publishing, Madison, WI.
Rigal, B. (2023). autograd-minimize. Retrieved from https://github.com/brunorigal/autograd-minimize.
Thygesen, U. H. (2023). Stochastic Differential Equations for Science and Engineering. Chapman and Hall/CRC.
Virtanen, P., Gommers, R., Oliphant, T. E., Haberland, M., Reddy, T., Cournapeau, D., Burovski, E., Peterson, P., Weckesser, W., Bright, J., van der Walt, S. J., Brett, M., Wilson, J., Millman, K. J., Mayorov, N., Nelson, A. R. J., Jones, E., Kern, R., Larson, E., Carey, C. J., Polat, İ., Feng, Y., Moore, E. W., VanderPlas, J., Laxalde, D., Perktold, J., Cimrman, R., Henriksen, I., Quintero, E. A., Harris, C. R., Archibald, A. M., Ribeiro, A. H., Pedregosa, F., van Mulbregt, P., & SciPy 1.0 Contributors (2020). SciPy 1.0: Fundamental algorithms for scientific computing in Python. Nature Methods, 17, 261–272.
Álvarez, J., Baratti, R., Tronci, S., Grosso, M., & Schaum, A. (2018). Global-nonlinear stochastic dynamics of a class of two-state two-parameter non-isothermal continuous stirred tank reactors. Journal of Process Control, 72, 1–16.
