2023 AIChE Annual Meeting

(317f) Towards Personalized Cold Plasma Treatments Using Safe Explorative Bayesian Optimization

Authors

Chan, K. J. - Presenter, University of California, Berkeley
Mesbah, A., University of California, Berkeley
Paulson, J., The Ohio State University
With the recent advances in computing, autonomous systems (those controlled by a pre-defined controller or policy) are becoming increasingly prevalent [1]. Now, some focus has shifted towards adapting these autonomous systems towards improved performance tailored for particular environments. Data-driven approaches such as reinforcement learning (RL) [2], [3] and Bayesian optimization (BO) [4], [5] have been used for this purpose. However, challenges remain in learning the appropriate adaptation of autonomous systems in safety-critical environments. At its core, adaptation in safety-critical environments is a constrained optimization problem. The problem often consists of black-box objectives and constraints that can only be queried as noisy observations. To solve such problems with RL or BO, a theoretically consistent way to account for constraints is to relax the constraint functions such that the feasible region has a high probability of containing the global solution [6]–[8]. However, these approaches are unable to ensure safe operation at every query, which is unacceptable under the notion of “safety-critical”. Alternatively, certain safe methods may force the query points to remain in the interior of a partially-revealed safety region [4], [9], which may result in unacceptable (and unquantified) performance losses. As such, safe methods can lose the explorative properties that often lead to more promising search regions.

This work presents SEBO (Safe Explorative Bayesian Optimization), a new safe BO method that avoids potential performance losses by reincorporating information gained by expanding the safe/feasible region [10]. As mentioned, standard safe BO may be prone to being overly conservative such that they may get stuck in the locally feasible region near the initial safe point. SEBO uses a relaxed formulation to widen the search space that more likely encapsulates the true optimum. Safety is ensured by projecting back to the estimated safe region, but at the same time, maximizing the potential to increase knowledge around the safe set in the direction of improvement. Thus, SEBO effectively incorporates directed information to explore the safe region(s).

We demonstrate SEBO for an exemplary application in personalized plasma medicine. Plasma medicine is an emerging field of study involving the use of cold atmospheric plasmas (CAPs) for a variety of medical treatments [11]. CAPs are a form of (partially) ionized gas that exist at near room temperature and atmospheric pressure, yet have high energy potential to induce low-level chemical, thermal, and electrical effects, making them amenable to medical applications [11]. Tailoring the plasma effects applied to a particular surface/subject is key to ensuring the efficacy of plasma treatments [12]. However, the underlying mechanisms of plasma-surface interactions are still an active area of research and can only be quantified for a population [13], [14]. Therefore, iterative improvements in automated treatments using BO will enable the personalization of CAP treatments, wherein ensuring (patient) safety is of the utmost importance. We compare the performance of SEBO in simulation to both aforementioned strategies of constrained optimization using BO. We demonstrate that it effectively combines elements from each strategy to increase exploration without violating safety.

[1] T. Zhang, Q. Li, C.-s. Zhang, H.-w. Liang, P. Li, T.-m. Wang, S. Li, Y.-l. Zhu, and C. Wu, “Current trends in the development of intelligent unmanned autonomous systems,” Frontiers of Information Technology & Electronic Engineering, vol. 18, pp. 68–85, 2017.

[2] M. K. Bothe, L. Dickens, K. Reichel, A. Tellmann, B. Ellger, M. Westphal, and A. A. Faisal, “The use of reinforcement learning algorithms to meet the challenges of an artificial pancreas,” Expert Review of Medical Devices, vol. 10, no. 5, pp. 661–673, 2013.

[3] Y. Wen, J. Si, A. Brandt, X. Gao, and H. H. Huang, “Online reinforcement learning control for the personalization of a robotic knee prosthesis,” IEEE Transactions on Cybernetics, vol. 50, no. 6, pp. 2346–2356, 2019.

[4] D. Krishnamoorthy and F. J. Doyle, “Safe Bayesian optimization using interior-point methods—applied to personalized insulin dose guidance,” IEEE Control Systems Letters, vol. 6, pp. 2834–2839, 2022.

[5] J. Shen, J. Hu, J. J. Dudley, and P. O. Kristensson, “Personalization of a mid-air gesture keyboard using multi-objective Bayesian optimization,” in Proceedings of the IEEE International Symposium on Mixed and Augmented Reality, 2022, pp. 702–710.

[6] J. A. Paulson and C. Lu, “COBALT: COnstrained Bayesian optimizAtion of computationaLly expensive grey-box models exploiting derivaTive information,” Computers & Chemical Engineering, vol. 160, p. 107700, 2022.

[7] C. Lu and J. A. Paulson, “No-regret Bayesian optimization with unknown equality and inequality constraints using exact penalty functions,” IFAC- PapersOnLine, vol. 55, no. 7, pp. 895–902, 2022.

[8] S. Pfrommer, T. Gautam, A. Zhou, and S. Sojoudi, “Safe reinforcement learning with chance-constrained model predictive control,” in Proceedings of the Learning for Dynamics and Control Conference, 2022, pp. 291–303.

[9] Y. Sui, A. Gotovos, J. Burdick, and A. Krause, “Safe exploration for optimization with Gaussian processes,” in Proceedings of the International Conference on Machine Learning, 2015, pp. 997–1005.

[10] K. J. Chan, J. A. Paulson, and A. Mesbah, “Safe explorative Bayesian optimization – Towards personalized treatments in plasma medicine,” 2023, Manuscript submitted for publication.

[11] M. Laroussi, “Plasma medicine: A brief introduction,” Plasma, vol. 1, no. 1, pp. 47–60, 2018.

[12] M. A. Hamburg and F. S. Collins, “The path to personalized medicine,” New England Journal of Medicine, vol. 363, no. 4, pp. 301–304, 2010.

[13] J.-M. Pouvesle and E. Robert, “Multimodal action of atmospheric pressure plasma jets for biological applications,” in ISPB 2017, 2017.

[14] A. D. Bonzanini, K. Shao, A. Stancampiano, D. B. Graves, and A. Mesbah, “Perspectives on machine learning-assisted plasma medicine: Toward automated plasma treatment,” IEEE Transactions on Radiation and Plasma Medical Sciences, vol. 6, no. 1, pp. 16–32, 2021.