(14a) Unveiling Latent Chemical Mechanisms: Achieving Robustness in Hybrid Models through Physics-Informed Regularization for Spatiotemporal Parameter Estimation in PDEs

In the realm of dynamic process modeling, differential equations, particularly PDEs, play a crucial role across various disciplines including chemical engineering. A key challenge in differential equations is solving the inverse problem, where unknown parameters are estimated from error-prone observed data [1,2]. Solution multiplicity is a common challenge in parameter estimation for PDE models where various parameters provide feasible solutions but they represent fundamentally different underlying processes or states of the system [3]. Despite the critical need to enhance model predictions and understand the dynamics of complex systems exhibiting spatiotemporal variations, there is a lack of effective and generalizable solutions in the literature. This gap is due to the challenges in capturing unknown latent mechanisms that cannot be explicitly described by mathematical functions. While there have been recent works that attempt to solve inverse problems like Bayesian models, iterative algorithms and recent advances in data-driven models like neural networks and physics-informed neural networks (PINNs) have been made, their application is limited in capturing the exact spatiotemporally varying nature of parameters. Traditional hybrid models which integrate data-driven models with first-principles model attempt to resolve this issue but struggle with issues concerning pre-training, and solution multiplicity [4].

To this end, we introduce a novel hybrid modeling approach that addresses these limitations by integrating regularization terms that reflect the natural gradients and patterns present in complex chemical systems into the loss function of hybrid models during training. This approach ensures that learned spatiotemporal parameters are statistically and physically grounded, reducing the risk of overfitting to non-physical data trends thereby enhancing the physical plausibility of the model outputs. A distinguishing feature of this work is the development of a two-phased training methodology, starting with a physics-informed 'warm-start' phase that guides the model towards a specific loss value, establishing a foundation for precise parameter convergence [5]. Following this, regularization is deactivated to resume standard model training, addressing the issue of pre-training in hybrid models effectively.

As a case study, this hybrid modeling framework is applied to a reaction-diffusion system to accurately estimate spatiotemporally varying diffusivity [6,7]. By employing the physics-informed regularization approach, we demonstrate the model's capability to handle the intricacies of pre-training, significantly improving the estimation of diffusivity across spatial and temporal dimensions [8]. The results confirm the approach's superiority in overcoming solution multiplicity, validated through rigorous comparisons against traditional models and leading to significant improvements in training and validation accuracies for predicting cell density. In conclusion, this study heralds a new class of hybrid models that not only boast remarkable precision and robustness but also remain aligned with underlying physical principles. By overcoming the challenges associated with solution multiplicity, this work marks a pivotal advancement in computational modeling for complex systems. These models effectively navigate the complexities of estimating spatiotemporally varying parameters, providing a powerful tool for probing into the latent mechanisms governing complex systems, and identifying the true dynamics when challenges of solution multiplicity exist.

References:

1. Aster, Richard C., Brian Borchers, and Clifford H. Thurber. Parameter estimation and inverse problems. Elsevier, 2018.
2. Tarantola, Albert, and Bernard Valette. "Inverse problems= quest for information." Journal of geophysics 50.1 (1982): 159-170.
3. Wang, Zhicheng, et al. "Solution multiplicity and effects of data and eddy viscosity on Navier-Stokes solutions inferred by physics-informed neural networks." arXiv preprint arXiv:2309.06010 (2023).
4. Shah, Parth, et al. "Deep neural network-based hybrid modeling and experimental validation for an industry-scale fermentation process: Identification of time-varying dependencies among parameters." Chemical Engineering Journal 441 (2022): 135643.
5. Klaučo, Martin, Martin Kalúz, and Michal Kvasnica. "Machine learning-based warm starting of active set methods in embedded model predictive control." Engineering Applications of Artificial Intelligence 77 (2019): 1-8.
6. Pahari, Silabrata, Parth Shah, and Joseph Sang-Il Kwon. "Unveiling Latent Chemical Mechanisms: Hybrid Modeling for Estimating Spatiotemporally Varying Parameters in Moving Boundary Problems." Industrial & Engineering Chemistry Research 63.3 (2024): 1501-1514.
7. Warne, David J., Ruth E. Baker, and Matthew J. Simpson. "Using experimental data and information criteria to guide model selection for reaction–diffusion problems in mathematical biology." Bulletin of Mathematical Biology 81.6 (2019): 1760-1804.
8. Pahari, Silabrata, Parth Shah, and Joseph Sang-Il Kwon. "Achieving robustness in hybrid models: A physics-informed regularization approach for spatiotemporal parameter estimation in PDEs." Chemical Engineering Research and Design (2024).