(394ag) A Novel Bayesian Approach for Data-Driven Hierarchical Sparse Modeling of Time-Delayed Process Systems

Data-driven model building continues to play an increasingly important role in the development of mathematical models required for process design, monitoring, optimization and control¹. Several approaches have been developed for data-driven model building for highly complex systems even when knowledge of the physics/chemistry of the is limited. Despite these successes, significant challenges confronting many data -driven modeling approaches include lack of interpretability, noise in the training data and unknown delays in each input and output variables used for model building.

In the past, building sparse data-driven models using well-defined basis functions has enabled the development of interpretable data-driven models that are parsimonious in model parameters and consequently, in the amount of data required for model building^2–4. Furthermore, the implementation of Bayesian inferencing in a two-step expectation maximization algorithm has facilitated successful development of highly predictive sparse dynamic models even when training data contains correlated noise^5,6. However, these existing methods do not take into consideration the presence of possible unknown delays in the input and output variables as would be expected for many real life process systems. In presence of time delays, the existing methods can lead to unsatisfactory sparse, interpretable data-driven models.

In this work, a novel approach is developed for building sparse hierarchical models from noisy data with unknown time delays. This approach draws motivation from the application of Predictive Coding (PC) for facilitating local computations as well as enhancing robustness and computational efficiency in the modeling of the Bayesian brain⁷. In the computational neuroscience literature, the hierarchical modeling of the Bayesian brain results in a structure in which top-down predictions from higher layers are matched with sensory data and incoming prediction errors from lower layers are inhibited⁸. Meanwhile, the Bayesian approach is used in Bayesian PC⁹ to infer the unknown (hidden) states, followed by employing a suitable learning algorithm for parameter estimation. Employing this strategy for hierarchical modeling of time-delayed systems will require complete knowledge of the delay in each variable but this information may not always be available. Hence, in this work, we propose two novel Hierarchical Bayesian Identification of Dynamic Sparse Algebraic Model (H-BIDSAM) algorithms that can optimally select the optimal basis functions from a set of candidate basis functions, optimally estimate the model parameters through Bayesian inference, and optimally estimate time delays in inputs/outputs. In the first method, an efficient bidirectional branch & bound algorithm is developed to solve the underlying MINLP problem, while the second algorithm includes a priori evaluation of the input variables for possible delays before employing the BIDSAM algorithm for model building.

The developed algorithms are evaluated for a plug flow reactor with and without spatial measurement of output variables. Model building is done using data to which temporal and cross-correlated noise has been added. It was found that the proposed approach gives parsimonious models with 10 -30% fewer parameters compared to state-of-the-art sparse modeling approaches while estimating the unknown time delays with >80% accuracy while yielding prediction RMSE below 5%.

References

1. Pantelides, C. C. & Renfro, J. G. The online use of first-principles models in process operations : Review , current status and future needs. Comput. Chem. Eng. 51, 136–148 (2013).
2. Sun, W. & Braatz, R. D. ALVEN : Algebraic learning via elastic net for static and dynamic nonlinear model identification. Comput. Chem. Eng. 143, 107103 (2020).
3. Brunton, S. L., Proctor, J. L., Kutz, J. N. & Bialek, W. Discovering governing equations from data by sparse identification of nonlinear dynamical systems. Proc. Natl. Acad. Sci. U. S. A. 113, 3932–3937 (2016).
4. Cozad, A., Sahinidis, N. V. & Miller, D. C. A combined first-principles and data-driven approach to model building. Comput. Chem. Eng. 73, 116–127 (2015).
5. Adeyemo, S. & Bhattacharyya, D. Optimal nonlinear dynamic sparse model selection and Bayesian parameter estimation for nonlinear systems. Comput. Chem. Eng. 180, 108502 (2024).
6. Mukherjee, A., Adeyemo, S. & Bhattacharyya, D. All‐nonlinear static‐dynamic neural networks versus Bayesian machine learning for data‐driven modelling of chemical processes. Can. J. Chem. Eng (2024) doi:10.1002/cjce.25379.
7. Friston, K. The free-energy principle: A unified brain theory? Nat. Rev. Neurosci. 11, 127–138 (2010).
8. Marino, J. Predictive coding, variational autoencoders, and biological connections. Neural Comput. 34, 1–44 (2021).
9. Whittington, J. C. R. An Approximation of the Error Backpropagation Algorithm in a Predictive Coding Network with Local Hebbian Synaptic Plasticity. 1262, 1229–1262 (2017).