(576d) Adaptive Reinforcement Learning Control for Chromatographic Separation Processes

Chromatographic separation processes are commonly employed for the purification of high-value biopharmaceuticals, ensuring the selective recovery of target compounds with high purity and yield. However, their inherent complexity, coupled with process variability and sensitivity to disturbances, presents significant challenges for real-time control (Rüdt et al., 2017). High-fidelity models, formulated using Partial Differential and Algebraic Equations (PDAEs), provide detailed descriptions of mass transfer and adsorption phenomena but impose high computational demands (Kumar et al., 2021). This limits their direct integration into conventional control frameworks, which often rely on simplified model approximations (Papathanasiou et al., 2016). As the industry moves towards autonomous and adaptive process operation, Reinforcement Learning (RL) has emerged as a powerful tool for real-time decision-making in complex systems (Andersson et al., 2023).

In this work, we develop an RL control framework for the twin-column Multicolumn Countercurrent Solvent Gradient Purification (MCSGP) process (Aumann & Morbidelli, 2007). An RL agent is trained directly on a high-fidelity model of the process, ensuring that the controller captures the full physicochemical complexity of the system. The controller is implemented from process start-up, continuously learning and adjusting operating parameters such as flowrates and switching times in response to real-time conditions. The performance of the RL controller is assessed under varying feed compositions and monitoring noise, evaluating its robustness and adaptability.

The RL framework employs a reward-driven optimisation approach, where the controller iteratively refines its policy to maximise key performance indicators (KPIs) such as product purity and process yield. The policy is learned through continuous interaction with the high-fidelity process model, using state observations, such as concentration profiles, to determine optimal control actions. The RL agent explores different control strategies, adjusting flowrates and switching times dynamically, while the reward function incentivises operational efficiency, minimising process duration, maximising process yield, and ensuring target purity. The performance of the RL control strategy is assessed across multiple operating scenarios, demonstrating its ability to maintain stability and enhance process resilience under uncertain conditions.

The results indicate that the RL controller effectively stabilises process performance, maintaining high product purity and yield while responding dynamically to process variations and disturbances. These findings highlight the potential of RL as an advanced control methodology for chromatographic separations, providing flexible and adaptable control strategies for complex chromatographic separation processes.

Acknowledgements

Funding from the UK Engineering & Physical Sciences Research Council (EPSRC) for the i-PREDICT: Integrated adaPtive pRocEss DesIgn and ConTrol (Grant reference: EP/W035006/1) and the IAA: Impact Acceleration Account (Grant reference: EP/X52556X/1) is gratefully acknowledged.

References

Andersson, D., Edlund, C., Corbett, B., & Sjögren, R. (2023). Adaptable control policies for variable liquid chromatography columns using deep reinforcement learning. Scientific Reports 2023 13:1, 13(1), 1–13.

Aumann, L., & Morbidelli, M. (2007). A continuous multicolumn countercurrent solvent gradient purification (MCSGP) process. Biotechnology and Bioengineering, 98(5), 1043–1055.

Kumar, V., Leweke, S., Heymann, W., von Lieres, E., Schlegel, F., Westerberg, K., & Lenhoff, A. M. (2021). Robust mechanistic modeling of protein ion-exchange chromatography. Journal of Chromatography A, 1660, 462669.

Papathanasiou, M. M., Avraamidou, S., Oberdieck, R., Mantalaris, A., Steinebach, F., Morbidelli, M., Mueller-Spaeth, T., & Pistikopoulos, E. N. (2016). Advanced control strategies for the multicolumn countercurrent solvent gradient purification process. AIChE Journal, 62(7), 2341–2357.

Rüdt, M., Brestrich, N., Rolinger, L., & Hubbuch, J. (2017). Real-time monitoring and control of the load phase of a protein A capture step. Biotechnology and Bioengineering, 114(2), 368–373.