(261d) A Superstructure Optimization Approach for Interpretable Reaction Kinetic Modeling with Small Datasets

This study proposes a novel approach for constructing interpretable reaction kinetic models from a small amount of data in multicomponent systems unlike ordinary machine learning methods. The approach utilizes the superstructure of all possible reaction pathways, constrained nonlinear programming, and sensitivity-based pruning method. The superstructure-based approach with the pruning method enables the identification of meaningful reaction pathways. The constrained nonlinear programming allows the incorporation of additional constraints based on prior knowledge, facilitating the training of models with small datasets. The sensitivity-based pruning method extracts key reactions by evaluating their importance that is calculated using the Hessian of the loss function in the training of models. The effectiveness of the proposed approach is demonstrated with complex reaction systems such as chlorination of ethylene and pyrolysis reaction of hydrocarbons. The method successfully modeled these intricate systems using significantly fewer data points compared to conventional approaches.

Kinetic modeling of chemical reactions is essential for understanding reaction characteristics, designing manufacturing processes, and optimizing operating conditions. In traditional kinetic modeling approach, chemists first hypothesize reaction mechanisms. Engineers then construct kinetic models and fit these models to experimental data. The process is typically repeated in a trial-and-error manner until a desired level of prediction accuracy is achieved. This approach, however, becomes impractical as the number of reactions increases. In contrast to the traditional first-principles approaches, Machine learning (ML) offers data-driven modeling techniques that do not require prior chemical knowledge. ML models, however, often lack interpretability because their parameters (e.g., weights and biases) have no physical meaning. To address this problem, Explainable AI (XAI) approaches [1] (e.g., SHAP, LIME, and Grad-CAM) are often used. While these methods provide the information of input-output relationship in ML models, they do not elucidate reaction pathways, which are essential for understanding the underlying chemical mechanisms.

Recent advances, such as Chemical Reaction Neural Networks (CRNNs [2]), address the issue by embedding reaction rate equations into ML models. This method allows parameters to have physical meanings, such as stoichiometric coefficients and rate constants. Despite these advancements, challenges persist in terms of data availability and model interpretability. For example, CRNNs sometimes produce reaction pathways that violate material balance, fail to yield stoichiometric coefficients as integers, and often require large datasets such as 3,000 data points even for a simple system which are difficult to obtain in real-world experiments due to time and cost constraints.

In this work, we propose a novel approach for constructing interpretable reaction kinetic models for multicomponent systems using small datasets. This approach integrates data-driven techniques with first-principles methods. The proposed framework comprises the following three steps.

1. Superstructure construction: The first step constructs the superstructure of all possible reaction pathways that satisfy atomic balance. In this step, we provide information on the molecular species considered to present in the system. Using this information, all possible reaction pathways that satisfy atomic balance are exhaustively enumerated up to a given reaction order.
2. Constrained nonlinear optimization: The second step solves a constrained nonlinear optimization problem that incorporates the kinetic models of the superstructural reaction pathways and prior knowledge to handle small datasets. The objective of the optimization is to minimize the mean square error (MSE) between experimental data and model predictions. The optimization is performed using Pyomo.DAE [3], which allows the inclusion of differential algebraic equations (DAEs) in the optimization problem. Besides the physics-based reaction kinetic models, knowledge-based constraints are introduced to supplement the limited data and enable robust modeling. For instance, constraints can be the monotonicity of temporal concentration changes or the sequence of experimental data points.
3. Sensitivity-based pruning: The third step prunes unnecessary reaction pathways using a sensitivity-based pruning method. Unlike magnitude-based methods that consider small parameter values as insignificant, the sensitivity-based approach determines the importance of parameters based on the sensitivity of the objective function to the rate constant. This approach is inspired by pruning methods in ML, such as Optimal Brain Surgeon (OBS [4]), which utilizes the Hessian of the objective function. In our method, we project the Hessian of Lagrangian of the constrained nonlinear problem onto the parameter space. The diagonal elements of the inverse Hessian matrix provide information of the uncertainty of the kinetic parameters.

Finally, 2^nd and 3^rd steps are repeated iteratively until the MSE increases to obtain a meaningful reaction set without sacrificing the prediction accuracy.

The proposed approach is demonstrated with the following two case studies.

Firstly, we applied this method to time-series data of ethylene chlorination, which was generated based on simulations using the model of [5] and confirmed that the method can identify the predefined reactions and reproduce the original reaction rate constants. The reaction system involves six components [C₂H₄, C₂H₃Cl, C₂H₄Cl₂, C₂H₃Cl₃, HCl, Cl₂] and consists of four reactions. The reaction rate constants for each reaction were arbitrarily set. Two datasets were generated with differing initial Cl₂ mole fractions, resulting in a total of 90 data points. Gaussian noise with a mean of 0 and a standard deviation of 0.01 was added to the data, and HCl was masked during data analysis to simulate realistic experimental conditions. In the first step of our framework, the maximum reaction order was restricted to 3 or less to reduce computational cost, resulting in the generation of 46 kinetic reactions. By performing 2^nd and 3^rd steps, we successfully extracted 4 correct reactions from this set and the rate constants for each reaction were correctly determined.

The second case study involves the pyrolysis reaction of hydrocarbons. The reaction system consists of 11 components. Five datasets were obtained from experiments by varying the amount of reagent added to the mixture. Fitting this data is particularly challenging because the total number of data points is only 56, while the reaction system includes 11 components. Furthermore, only 8 of these components can be observed, as 3 components are masked. To address this challenging problem, we employed a “no-overtaking constraint”. This constraint assumes that the concentration of each component increases or decreases monotonically with respect to the amount of reagent added, meaning that the order of the components does not change or "overtake" one another in the observed time window. In the first step, a total of 550 reaction pathways were generated. From this set, only the 40 pathways that fit the experimental data were selected automatically. While conventional CRNN required around 3,000 data points to model a reaction system with 5 components, the proposed method successfully modeled a system with 56 data points with 11 components even with three components are masked.

In conclusion, we have demonstrated an innovative method for constructing interpretable kinetic models from sparse experimental datasets in multicomponent systems. Our approach integrates the superstructure of all possible reaction pathways, constrained nonlinear optimization with chemical knowledge, and sensitivity-based pruning to construct models from limited data. We applied this method to the chlorination of ethylene as well as the pyrolysis reaction of hydrocarbons, successfully identifying key reactions and rate constants with significantly fewer data points compared to conventional ML approaches. The proposed reaction modeling framework enables efficient modeling and enhances the understanding of reaction systems, even in data-scarce situations, which are often caused by experimental constraints such as time, cost, technical limitations, safety concerns, and selective data collection.

【References】

[1] Barredo Arrieta, A.; Díaz-Rodríguez, N.; Del Ser, J.; Bennetot, A.; Tabik, S.; Barbado, A.; Garcia, S.; Gil-Lopez, S.; Molina, D.; Benjamins, R.; Chatila, R.; Herrera, F. Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI. Inf Fusion 2020, 58 (1), 82–115.

[2] Ji, W.; Deng, S. Autonomous Discovery of Unknown Reaction Pathways from Data by Chemical Reaction Neural Network. J. Phys. Chem. A 2021, 125 (5), 1082–1092.

[3] Nicholson, B. L.; John Daniel Siirola; Watson, J.-P.; Zavala, V. M.; Biegler, L. T. Pyomo.dae: A Modeling and Automatic Discretization Framework for Optimization with Differential and Algebraic Equations. 2017, 10 (2), 187–223.

[4] Hassibi, B.; Stork, D. G. Second Order Derivatives for Network Pruning: Optimal Brain Surgeon. Adv. Neural Inf. Process. Syst. 1993, 5, 164–171.

[5] Shokrollahi Yancheshmeh, M. S.; Seifzadeh Haghighi, S.; Gholipour, M. R.; Dehghani, O.; Rahimpour, M. R.; Raeissi, S. Modeling of Ethane Pyrolysis Process: A Study on Effects of Steam and Carbon Dioxide on Ethylene and Hydrogen Productions. Chem. Eng. J. 2013, 215-216, 550–560.