(2gl) A-Priori Theory-Informed Training of Artificial Neural Networks for Prediction of Chemical Reactivity

Artificial neural networks (ANN) are a promising technology for the fast prediction of physicochemical properties of arbitrary molecules. Especially, the ANNs for combustion properties prediction provides a useful guideline for pre-screening alternative fuel candidates for a targeted energy system. However, the training ANNs for combustion properties are hindered by the scarcity of data (~ a few hundred), coming from the limited measurement range of combustion experiments. Such a challenge may be overcome when the ANNs are carefully trained with support from theoretical kinetics predictions. My research interest is i) to discover the key molecular features related to the combustion properties using experimental and theoretical kinetics studies, and ii) to train ANNs based on the recognized molecular features for better prediction accuracy with the limited experimental database.

My previous research at National Renewable Energy Laboratory explored the key molecular features determining the cetane number (CN) and yield sooting index (YSI) of ether-containing molecules, representing the autoignition and soot characteristics of fuel, respectively. We executed flow reactor experiments of five ether molecules with varying molecular structures (one linear ether, one branched ether, and three cyclic ethers), from which we found a correlation between the combustion properties (CN and YSI) and the molecular structure of product species. Theoretical kinetics studies using ab-initio calculations elucidated the reaction pathways from fuel molecules to each product species. Consequently, we could recognize the key molecular features in fuels determining their CN ad YSI: the location of an oxygen atom, ring size, and carbon type (primary to quaternary). The key molecular features were then used as the input parameters of the multivariate regression model for CN and YSI prediction of ether-containing molecules, showing remarkable accuracy (R²=0.91 for CN and 0.95 for YSI) despite the simplicity of the regression model.

The kinetics study on the CN of ether-containing molecules inspired me to develop the ANNs predicting the CN of arbitrary molecules based on the aforementioned key molecular features. Graph neural networks (GNNs) were employed as it inherently considers the molecular structure as input parameters. Additionally, the number of hydrogen bond donors and acceptors were also provided as input parameters to GNNs since the theoretical analysis revealed that the intramolecular hydrogen bonds play an important role in determining the equilibrium constants of key reaction steps. Training the GNNs to the limited number of CN data (630 single compounds), we showed that the accuracy of GNNs is remarkably improved when the kinetics-related molecular features were used as input parameters. Moreover, one of the beauties of GNNs is that it outputs the relative importance of each heavy atom (C and O in this case) in the molecule to the CN determination, enabling the discovery of other molecular features where the further kinetics study is required.

Currently, my research at Argonne National Laboratory is devoted to modeling gas-phase reaction kinetics, which is of particular interest to the atmospheric and combustion community. I'm interested in the kinetics simulation of the non-thermal reactions that prevails at high temperature and low-pressure condition. In this regard, my recent paper discussed the relevance of the non-thermal reactions to the combustion kinetics simulation, where the rate constants of thermal and non-thermal reactions were calculated from the state-of-art theory of electronic structure and collisional energy transfer. Our research revealed that the influence of non-thermal reactions on the flame simulation is substantial (up to ~ 15 % effect on flame speed prediction), which is sensitive to collisional energy transfer and the rate of competing reactions. The follow-up paper will discuss how the extent of non-thermal reactions depends on the molecular structure of participating free radicals. I'd like to apply the comprehensive knowledge from this theoretical kinetics study to the ANNs for rapid predictions of the non-thermal reaction rates, which will significantly improve the fidelity of combustion simulations for novel propulsion systems at high temperatures and sub-atmospheric pressure.

The idea of kinetics-informed training of ANNs can be applied to various research areas involving chemical kinetics. My future research interest includes its application to discovery of pharmaceuticals, energy storage materials, and novel catalysts. I believe my solid background and skillsets in both experimental and theoretical kinetics will be useful for pursuing the proposed ideas that have broad relevance to multiple research areas.

Research experience:

• Research assistant, Seoul National Univ. 2014 – 2020 (advisor: Prof. Han Ho Song)
  Kinetics experiments using a rapid compression machine.
• Postdoctoral researcher, National Renewable Energy Lab. 2020 – 2021 (advisor: Prof. Seonah Kim)
  Electronic structure theory, Graph neural network
• Postdoctoral researcher, Argonne National Lab. 2021 – present (advisor: Dr. Raghu Sivaramakrishnan)
  Electronic structure theory, Theoretical Kinetics, Kinetics modeling, Kinetics simulation

Teaching Interests

As an engineer with a background in both experiment and theoretical research, I believe scientific research provides more copious insight into physical phenomena when the experimental and theoretical approaches compensate for each other. However, as science and technology evolve, it becomes hard for the students to be trained in both experimental and theoretical approaches, which prevents them from having a balanced perspective on the strengths and weaknesses of each. In this regard, one of my teaching interests is to deliver how to analyze the same physical phenomena from both experimental and theoretical methodologies.

Another teaching interest of mine is to emphasize the social responsibility of engineers. I believe the ultimate goal of engineering is to make human life more convenient and safe. However, engineering education in reality focuses more on the physics and mathematics of systems, giving students little chance to reflect on how to utilize their knowledge for society. Providing an experience of solving problems in a public community will let the student think about the engineer's role in society.

Based on the abovementioned teaching interests, I suggested two courses as below.

Subject 1 – Combustion: Understanding the difference between experiment and theory

Combustion phenomena involve thousands of chemical reactions with the conversion of chemical energy into thermal or mechanical energy, which has been the topic of interdisciplinary research between engineers and scientists. This complex nature makes the combustion course suitable for teaching experimental and theoretical methodology, allowing students to understand combustion science comprehensively. In this regard, I would like to devote about 3 out of 15 weeks of curriculum to a lab course emphazising combustion fundamentals. For example, the students will be taught to execute one of the simplest experiments using a Bunsen burner. They will learn how to quantitatively measure the flame length and angle from the flame image and how they depend on the fuel/air flow rate. This experimental course will teach the student how the experimental data support the combustion theory and where the uncertainty of the measurement comes from. And over the next 3 weeks, I'd like to teach them how to simulate the same combustion system numerically. The students will learn how to apply the finite element method to the flame simulation and how many grid points are optimal for achieving reliable results with minimal computational cost. After completing the practices on both experiment and numerical analysis, I will let the student compare the two results and discuss the strengths and weaknesses of each methodology. I expect this class will help the student have a balanced perspective on the field of experiments and numerical analysis.

Subject 2 – Sustainable Energy System Design: Return the engineering to society.

As an advanced course for engineering, I would like to open a lecture on how to design a sustainable energy system. The lecture will provide a practice class on how to model the energy system with the open-source software and how to evaluate the thermal efficiency and the greenhouse gas (GHG) emission from the system. Ultimately, this lecture will let the student analyze the sustainability of the engineering process in countries without sufficient energy sources. For example, around 13 % of the world's population cannot access electricity, and this course will provide an opportunity to design a sustainable energy system for them. The student will be able to develop a simple system such as a portable generator with a solar panel or an efficient boiler burning livestock waste. Then, I expect them to validate the feasibility and sustainability of the idea using a computational model and discuss what kinds of technical challenges are expected. Ultimately, the experience of applying their knowledge to solve the engineering problem in public will allow the students to think about the engineer's role in society.