(11f) A Computational Study on the Benefits of Decision-Focused Surrogate Modeling

Optimization using rigorous computational models of chemical systems is often difficult due to the complex underlying physics of such systems. A common strategy for solving these problems, especially in real-time applications, is to develop surrogate models of reduced computational complexity. While surrogate modeling has a long tradition in process systems engineering [1], recent focus has been on data-driven surrogate modeling methods that take advantage of advances in machine learning [2, 3]. In data-driven surrogate modeling, one typically replaces either a part of or the entire optimization model with a machine learning model. However, as we show in our previous work [4], these standard approaches have several disadvantages due to which they do not always lead to a low decision error, which is defined as the difference between the optimal solutions of the original and the surrogate optimization models.

To overcome the drawbacks of the standard methods, we developed a data-driven inverse optimization (IO) approach to construct surrogate optimizers of reduced complexity [5]. Our IO approach allows direct decision-focused learning, i.e., the models are trained to obtain (almost) the same optimal solutions as the original optimization models. Furthermore, in contrast to traditional machine learning models, IO allows the incorporation of domain knowledge in the form of explicit constraints and tends to be more data-efficient.

In this work, we validate our framework through numerical experiments involving the real-time optimization of common chemical processes such as chemical reactors, and heat exchanger networks. We test our decision-focused surrogate modeling method against several standard data-driven surrogate modeling approaches. We find that with our framework, even simple surrogate models that are linear in the decision variables result in high out-of-sample prediction accuracies. This is because, while nonconvex functions cannot generally be approximated well with linear functions, our approach allows to “transfer” the nonconvexity from the decision variable space to the input space, which is enough for the surrogate model to learn the key features of the original model. In contrast, we find that optimization models obtained with standard surrogate modeling approaches struggle to match the true optimal solutions even when highly sophisticated machine learning frameworks such as deep learning are used.

References:

1. 1. Bhosekar, A., & Ierapetritou, M. (2018). Advances in surrogate based modeling, feasibility analysis, and optimization: A review. Computers & Chemical Engineering, 108, 250-267.
2. 2. Kim, S. H., & Boukouvala, F. (2020). Machine learning-based surrogate modeling for data-driven optimization: a comparison of subset selection for regression techniques. Optimization Letters, 14(4), 989-1010.
3. 3. Cozad, A., Sahinidis, N. V., & Miller, D. C. (2014). Learning surrogate models for simulation‐based optimization. AIChE Journal, 60(6), 2211-2227.
4. 4. Gupta, R., & Zhang, Q. (2021). A data-driven inverse optimization approach to learning surrogate optimizers. 2021 AIChE Annual Meeting. AIChE
5. 5. Gupta, R., & Zhang, Q. (forthcoming). Decision-focused surrogate modeling with feasibility guarantee. Proceeding of the 14^th International Symposium on Process Systems Engineering.