(373n) Open-Loop Simulation Surrogates for Closed-Loop Global Optimization of Sustainable Processes

Today’s chemical sector is responsible for more than 2.0 Gt CO₂ yearly emissions and is expected to reach around 4.5 Gt CO₂ in 2050 [1]. To limit global warming to 1.5 °C above pre-industrial levels with limited or no overshoot, net-zero global CO₂ emissions need to be achieved by mid-century [2]. Reaching carbon neutrality in the chemical sector is challenging because of its overdependence on fossil resources as feedstock (i.e., raw materials for chemical synthesis) and energy sources in chemicals production; hence, it is often referred to as a hard-to-abate sector. Ultimately, the carbon cycle must be closed, which might involve to some extent deploying carbon capture and storage with current fossil-based infrastructure and moving away from fossil to renewable resources, e.g. captured CO₂, recycled chemical products at the end-of-life phase, and biomass [1]. Connected to the latter, there exists a strong need for potentially rethinking the building blocks of a more sustainable chemical industry that can produce platform chemicals (e.g., ammonia, methanol, olefins, and aromatics) from which the whole chemical value chain can be derived [3]. This big change in the chemical sector will inevitably require huge efforts from the Process Systems Engineering (PSE) community, among many others, to optimally design the future, sustainable industry, helping to overcome economic, political, and societal barriers while alleviating environmental pressures.

Despite the impressive accomplishments the PSE community has achieved thus far in developing a more energy- and resource-efficient chemical industry, it is possible that the existing tools for process simulation and optimization will not be enough to enable future endeavors in the transition toward carbon neutrality. Joint efforts from molecular to plant level are required, going from catalyst, equipment, and process design all the way to their global implications [4]. In this work, we focus on optimal process design, which requires, for more informed decision-making, a comprehensive understanding of the underlying systems here attributed to rigorous process simulators. The design task can be formulated as an optimization problem, aimed at minimizing (or maximizing) an objective function (e.g., costs and environmental impacts) while satisfying the design constraints. The desired accurate models and resulting small-scale optimization problems come at the price of additional challenges that arise from embedding process simulation into such design problems, including (i) lack of analytical formulation for deterministic global optimization [5]; (ii) noisy simulations that compromise derivative approximations [6]; (iii) unconverged simulation that may implicate on premature termination of the optimization algorithm [7]; and high computation expenses that hinder the more complex analysis like multi-objective optimization [8].

The most common optimization approaches to rigorous process design involve optimizing the simulated process flowsheet either with (approximated) derivate-based deterministic solvers, facing issues as explained above, or by employing derivative-free or metaheuristic algorithms such as genetic algorithm or particle swarm [9]. The latter can naturally optimize black-box optimization problems, as they only need to have access to information about the objective function for any given selection of design variables. However, they lack optimality guarantees and are frequently much slower than their derivative-based counterparts.

More recently, the use of machine learning models in surrogate-based optimization frameworks has offered potential solutions to overcoming the challenges of dealing with black-box simulations [10]. The numerous approaches vary mainly due to the surrogacy level and the surrogate choice. Regarding the former, a surrogate that seeks to replace the whole system (flowsheet) is referred to as a plant, system, or black-box level surrogate model. In contrast, the surrogate model can be hybridized with first-principle equations [11] at a unit operation or even property level. As for the latter, surrogates can be selected based on their predictive performance or easiness of optimization and some representatives include polynomial regression, neural networks, gaussian processes, radial basis functions, decision trees, etc. One known issue in the choice of surrogate models is that some, often powerful in their predictive power, are hungry for data, and generating large amounts of data with process simulators is notably challenging.

Attempting to solve the above-mentioned potential issues with simulation and surrogate-based approach, especially in terms of computation time for large datasets and noise in the calculations due to numerical convergence, we propose a new surrogacy level, hereby called open-loop plant level. It consists of generating data from the simulation in an open loop, i.e., without closing recycles or other computationally intensive and failure-prone iterative calculations. Consequently, new degrees of freedom, beyond the design variables, are introduced (e.g., temperature, pressure, and component molar flow of the tearing streams leaving the recycle). Similar ideas have been considered in different contexts. For example, in derivative-based [12] or hybrid surrogate-simulation [9] optimization, the simulation iterative calculations can be turned off and added as constraints in the optimization problem. Differently, here, we fit surrogate models to the process parameters that are being adjusted by the iterative calculations and add constraints to the surrogate optimization problem to enforce closing the recycle loops and iterative process specifications. To exemplify, in the context of recycle loops, the tearing stream’s degrees of freedom must be equal to those from the stream that would be entering the recycle.

All in all, the optimization framework employed in the present research employs the following steps: (i) initial open-loop sampling procedure, (ii) building the surrogate models, (iii) solving the closed-loop surrogate optimization problem, (iv) resampling at the surrogate optimum, and, optionally, (v) recalibration of surrogate models. The choice of surrogate model for the present work was the ReLU (rectified linear unit) neural networks using the Python package OMLT [13], for their ability to be expressed as a MILP (mixed-integer linear programming) and solved to some extent global optimality with linear solvers (e.g., CPLEX and Gurobi).

Applying the proposed methodology in the design of several sustainable chemical processes, we found that by simulating in open-loop (with iterative calculations deactivated), the simulation time was reduced by orders of magnitude; therefore, allowing the generation of a bigger dataset to train the data-driven models in comparable time. That also allowed noise-free simulations, whereas its closed-loop counterpart could introduce considerable noise even for small convergence tolerances. The large datasets generated in the open-loop plant-level surrogacy resulted in ReLU neural networks that could fit well both the train and test set with low values of mean squared error. The closed-loop simulation of the open-loop surrogate global optimum showed that constraints could enforce the iterative calculations (i.e., closed recycle loop) within tolerances. The present approach could outperform some of the state-of-the-art derivative-based, derivative-free, metaheuristics, and closed-loop surrogate-based optimization methods, yet the best approach will always depend on the specific problem at hand. Additionally, the proposed method can be used to detect infeasibilities due to the violation of first principles (i.e., not connected to numerical issues) early on without the need to spend time checking whether convergence is attained.

References

[1] Gabrielli, P. et al. Net-zero emissions chemical industry in a world of limited resources. One Earth 6, 682–704 (2023).

[2] Calvin, K. et al. IPCC, 2023: Climate Change 2023: Synthesis Report. https://www.ipcc.ch/report/ar6/syr/ (2023) doi:10.59327/IPCC/AR6-9789291691647.

[3] Galán-Martín, Á. et al. Sustainability footprints of a renewable carbon transition for the petrochemical sector within planetary boundaries. One Earth 4, 565–583 (2021).

[4] González-Garay, A. et al. Plant-to-planet analysis of CO 2 -based methanol processes. Energy Environ. Sci. 12, 3425–3436 (2019).

[5] Boukouvala, F. & Floudas, C. A. ARGONAUT: AlgoRithms for Global Optimization of coNstrAined grey-box compUTational problems. Optim. Lett. 11, 895–913 (2017).

[6] Caballero, J. A. & Grossmann, I. E. An algorithm for the use of surrogate models in modular flowsheet optimization. AIChE J. 54, 2633–2650 (2008).

[7] Javaloyes-Antón, J., Kronqvist, J. & Caballero, J. A. Simulation-based optimization of distillation processes using an extended cutting plane algorithm. Comput. Chem. Eng. 159, 107655 (2022).

[8] Santos, L. F., Costa, C. B. B., Caballero, J. A. & Ravagnani, M. A. S. S. Multi-objective simulation–optimization via kriging surrogate models applied to natural gas liquefaction process design. Energy 262, 125271 (2023).

[9] Quirante, N., Javaloyes-Antón, J. & Caballero, J. A. Hybrid simulation-equation based synthesis of chemical processes. Chem. Eng. Res. Des. 132, 766–784 (2018).

[10] Bhosekar, A. & Ierapetritou, M. Advances in surrogate based modeling, feasibility analysis, and optimization: A review. Comput. Chem. Eng. 108, 250–267 (2018).

[11] Jog, S., Vázquez, D., Santos, L. F., Caballero, J. A. & Guillén-Gosálbez, G. Hybrid analytical surrogate-based process optimization via Bayesian symbolic regression. Comput. Chem. Eng. 182, 108563 (2024).

[12] Biegler, L. T., Grossmann, I. E. & Westerberg, A. W. Systematic Methods of Chemical Process Design. (Prentice Hall PTR, 1997).

[13] Ceccon, F. et al. OMLT: Optimization & Machine Learning Toolkit. J. Mach. Learn. Res. 23, 1–8 (2022).