Conversely, hard-constrained model approaches guarantee exact constraint satisfaction. Various methodologies have been proposed to enforce the exact satisfaction of physical constraints. Prediction and completion approaches enforce equality and inequality (non)-linear constraints by predicting a subset of variables. The predicted subset of variables has the same codimension as the number of equality constraints. This is followed by solving a system of (non)-linear equations to predict the remaining variables (3). Another approach proposed recently is to train constrained neural networks using nonlinear programming solvers to guarantee the satisfaction of physical laws (4). However, both approaches become computationally expensive due to their reliance on external nonlinear optimization solvers, and they scale poorly with deep networks or large training datasets.
Chen and co-workers proposed appending a projection layer to the neural network to enforce linear equality constraints and mitigate the costs associated with external nonlinear optimization solvers (5). The parameters of the projection layer can be obtained via the analytical solution of KKT conditions, which is computationally cheap. An extension of this approach was proposed, which adapted Picard’s successive approximation method for solving nonlinear partial differential equations to enforce nonlinear enthalpy balances (6).
There has been a growing interest in embedding physics-informed neural networks (PINNs) in mathematical optimization problems in engineering, and several comparative studies have been conducted. Casas and co-workers compared different strategies to embed soft-constrained physics-informed neural networks via its algebraic formulation or as a gray-box within the optimization model for model predictive control problems (7). Other studies considered the comparison of soft-constrained PINNs with vanilla neural networks for optimal power flow problems and reported that predictions from PINNs offer better worst-case guarantees, particularly in data-limited settings (8). Expectedly, violations in physical laws are observed for soft-constrained PINNs, albeit the constraint violations are lower relative to vanilla neural networks (8). However, none of these studies consider hard-constrained neural networks in their comparison.
In this work, we compare hard-constrained PINNs to soft-constrained PINNs and vanilla neural networks as surrogate models within larger chemical process optimization problems. We individually consider the enforcement of both linear and nonlinear equality constraints in our comparison. We conduct the comparison for the case of replacing fixed-bed catalytic reactor models with neural network surrogates in chemical processes. We consider the data requirements, the physical validity of optimal solutions with surrogates, and the tractability of each model for our case studies. We observe that embedding hard-constrained brings significant prediction benefits with exact satisfaction of physical laws, particularly in data-scarce settings. However, this benefit comes at a slightly higher computational cost for optimization compared to soft-constrained PINNS and vanilla neural networks.
References: