2022 Annual Meeting
(362f) Development of Bayesian Machine Learning Algorithms for Optimal Nonlinear Model Selection and Parameter Estimation from Noisy Data of Unknown Characteristics
Recently, several authors have employed sparse regression for succinct representation of dynamical systems through optimal selection of basis function from several candidate basis functions[2][3]. However, the algorithms are applicable to the data without noise and therefore the efficacy of the developed algorithms were tested using simulated data. Furthermore, the candidate basis functions were rather limited. However, most real-life process data are corrupted with noise with unknown characteristics. Besides, there may be colored noise present in the data.
In this work, we develop a Branch and Bound algorithm [4] for subset selection using an information criteria that rewards model fitness while penalizing model size and model complexity. Large number of basis functions are considered including polynomial, logarithmic, and exponential functions, and interaction terms between inputs and outputs including time delays this leading to a large-scale combinatorial optimization problem. A desired number of hierarchically ranked candidate models are obtained leaving the user with the choice of selecting any top rank model that yields succinct representation of the system with explainable basis functions. Parameters are estimated by employing Bayesian inferencing by maximizing the likelihood of the estimated model parameters. The two-step approach also estimates the unknown noise characteristics in an outer loop conditioned on available data. The method facilitates incorporating prior user belief of model parameters. Based on an expectation maximization algorithm, the algorithm yields simultaneous estimation of the noise covariance in addition to the model parameters.
The proposed algorithm is applied to the simulated data for a reactor-separator system and also to the operational data from an industrial boiler. The algorithm shows promise for flexible and succinct model development for highly nonlinear systems from data with correlated noise and unknown noise characteristics.
References
[1] T. Bankole and D. Bhattacharyya, âExploiting connectivity structures for decomposing process plants,â J. Process Control, vol. 71, pp. 116â129, 2018, doi: 10.1016/j.jprocont.2018.09.002.
[2] B. Bhadriraju, M. S. F. Bangi, A. Narasingam, and J. S. Il Kwon, âOperable adaptive sparse identification of systems: Application to chemical processes,â AIChE J., vol. 66, no. 11, pp. 1â15, 2020, doi: 10.1002/aic.16980.
[3] N. M. Mangan, J. N. Kutz, S. L. Brunton, and J. L. Proctor, âModel selection for dynamical systems via sparse regression and information criteria,â Proc. R. Soc. A Math. Phys. Eng. Sci., vol. 473, no. 2204, 2017, doi: 10.1098/rspa.2017.0009.
[4] A. A. H. Land and A. G. Doig, âAutomatic Method of Solving Discrete Programming Problemsâ. JSTOR vol. 28, no. 3, pp. 497â520, 1960.