(59i) A Novel Recurrent Neural Network for Hydroprocessing Unit Modeling Using Neural Circuit Policies and Attention-Based Encoder-Decoder

Hydroprocessing is a crucial process in the petrochemical industry, which consists of hydrotreating and hydrocracking. Hydrotreating removes impurities from crude oil, while hydrocracking converts heavy molecules into lighter and more valuable products. Both of them typically take place in a hydroprocessing unit and use high-pressure hydrogen gas as well as catalysts to produce cleaner and more valuable products [1].

Accurate predictive models are important for controlling and optimizing hydroprocessing unit production rates and operating costs. They help identify potential bottlenecks and prevent costly downtime and maintenance issues [2]. Establishing physical models for hydroprocessing units is challenging because hydroprocessing units involve sophisticated chemical reactions, heat transfer mechanisms, and thermodynamics [3]. Alternatively, data-driven models are preferred since they can be constructed using historical data without a complete understanding of the system's underlying physics. Also, they are more flexible, adaptable, and computationally efficient compared to physical models [4]. Therefore, data-driven models are more suitable for accurately predicting hydroprocessing unit behavior than physical models.

Among numerous data-driven modeling techniques, recurrent neural networks (RNNs) are a popular data-driven modeling technique for dynamic systems. Unlike feed-forward neural networks, RNNs have feedback loops that use previous input information to compute the hidden state in the current time step. The hidden state acts as a memory that captures dynamic behavior that enables RNNs to outperform feed-forward neural networks when modeling sequential data. The vanilla RNN, Long Short-Term Memory (LSTM), and Gated Recurrent Unit (GRU) are commonly used RNNs. While the vanilla RNN has the short-term memory problem [5], the LSTM and GRU use memory cells and gates to selectively store or forget information that enables them to handle longer sequences. The LSTM is capable of capturing both short and long-term dependencies, while the simpler structure of the GRU makes it faster to train and more computationally efficient [6].

In addition to the above-mentioned RNNs, neural circuit policies (NCPs) are bio-inspired RNNs inspired by the nervous system of the C. elegans nematode (a small worm-like species) [7]. An NCP consists of four layers of neurons: sensory neuron, interneuron, command neuron, and motor neuron. NCPs have feed-forward connections between layers and recurrent connections among command neurons. Each neuron in an NCP is a liquid time-constant (LTC) neuron which is an ordinary differential equation (ODE) system with excellent performance in modeling system dynamics [8]. Compared to ordinary RNNs such as LSTM and GRU, NCPs are computationally more efficient, structure-wise more compact, and require fewer neurons. NCPs have been applied to autonomous driving [7], soft-sensor modeling [9], autonomous flight [10], etc.

A time-series model capable of handling different-length input and output sequences is required to accurately model the behavior of a hydroprocessing unit. The encoder-decoder RNN (ED-RNN) [11] is well-suited for this purpose, which consists of an encoder RNN and a decoder RNN. The encoder RNN encodes an input sequence into a fixed-length vector, and the decoder RNN decodes the vector into an output prediction. The attention mechanism [12] can be further added to the ED-RNN to be the attention-based ED-RNN (A-ED-RNN), to improve its performance and computational efficiency by allowing the model to focus on important parts of the input sequence. The ED-RNN and A-ED-RNN have been successfully applied in chemical engineering for process monitoring, fault detection & diagnosis, soft sensor modeling, and predictive maintenance [13].

This study introduces ED-RNN and A-ED-RNN frameworks for modeling a real industrial hydroprocessing unit and predicting the diesel and jet production rates of the hydroprocessing unit. Moreover, NCPs are incorporated into these frameworks to create two novel RNN methods, namely NCP-ED-RNN and NCP-A-ED-RNN. Based on our experiment, the two methods are both effective in accurately predicting the production rates of the hydroprocessing unit, with NCP-A-ED-RNN being much more memory-efficient than other RNN methods, under the same predictive performance. This is attributed to the attention layer and NCP cells that make the NCP-A-ED-RNN more compact and require fewer neurons and trainable parameters. Since the NCP-A-ED-RNN is more memory-efficient, it is well-suited for deployment in industrial applications where memory resources are typically limited. Also, since the NCP-A-ED-RNN is more compact, it may have better model interpretability [7] which allows model developers to gain a better understanding of the rationale behind the model's predictions in comparison to other RNN methods.

References

[1] P. R. Robinson and G. E. Dolbear, "Hydrotreating and hydrocracking: fundamentals," Practical advances in petroleum processing, vol. 1, pp. 177-217, 2006.

[2] D. Ghosh, J. Moreira, and P. Mhaskar, "Application of data-driven modeling approaches to industrial hydroprocessing units," Chemical Engineering Research and Design, vol. 177, pp. 123-135, 2022.

[3] E. Aydın et al., "Dynamic modeling of an industrial diesel hydroprocessing plant by the method of continuous lumping," Computers & Chemical Engineering, vol. 82, pp. 44-54, 2015.

[4] H. Tao et al., "Artificial intelligence models for suspended river sediment prediction: state-of-the art, modeling framework appraisal, and proposed future research directions," Engineering Applications of Computational Fluid Mechanics, vol. 15, no. 1, pp. 1585-1612, 2021.

[5] D. K. Bilal, M. Unel, L. T. Tunc, and B. Gonul, "Development of a vision based pose estimation system for robotic machining and improving its accuracy using LSTM neural networks and sparse regression," Robotics and Computer-Integrated Manufacturing, vol. 74, p. 102262, 2022.

[6] R. Rouhi Ardeshiri and C. Ma, "Multivariate gated recurrent unit for battery remaining useful life prediction: A deep learning approach," International Journal of Energy Research, vol. 45, no. 11, pp. 16633-16648, 2021.

[7] M. Lechner, R. Hasani, A. Amini, T. A. Henzinger, D. Rus, and R. Grosu, "Neural circuit policies enabling auditable autonomy," Nature Machine Intelligence, vol. 2, no. 10, pp. 642-652, 2020.

[8] R. Hasani, M. Lechner, A. Amini, D. Rus, and R. Grosu, "Liquid time-constant networks," in Proceedings of the AAAI Conference on Artificial Intelligence, 2021, vol. 35, no. 9, pp. 7657-7666.

[9] J. Loy-Benitez, S. Tariq, H. T. Nguyen, U. Safder, K. Nam, and C. Yoo, "Neural circuit policies-based temporal flexible soft-sensor modeling of subway PM2. 5 with applications on indoor air quality management," Building and Environment, vol. 207, p. 108537, 2022.

[10] P. Tylkin et al., "Interpretable autonomous flight via compact visualizable neural circuit policies," IEEE Robotics and Automation Letters, vol. 7, no. 2, pp. 3265-3272, 2022.

[11] F. Li et al., "A hierarchical temporal attention-based LSTM encoder-decoder model for individual mobility prediction," Neurocomputing, vol. 403, pp. 153-166, 2020.

[12] Z.-L. Ni, G.-B. Bian, Z.-G. Hou, X.-H. Zhou, X.-L. Xie, and Z. Li, "Attention-guided lightweight network for real-time segmentation of robotic surgical instruments," in 2020 IEEE international conference on robotics and automation (ICRA), 2020, pp. 9939-9945: IEEE.

[13] M. M. Aliabadi, H. Emami, M. Dong, and Y. Huang, "Attention-based recurrent neural network for multistep-ahead prediction of process performance," Computers & Chemical Engineering, vol. 140, p. 106931, 2020.