(170f) Crystalgpt: Enhancing System-to-System Model Interchangeability in Crystallization Prediction and Control Using a Time-Series-Transformer Model

Given the complexity of first-principled crystallization models, often a surrogate model is constructed to provide a computationally leaner digital twin for process monitoring and control. Although various state-of-the-art (SOTA) machine learning (ML) have been used as surrogates with high prediction accuracy, they often are very system-specific, and cannot easily be repurposed from system `A' to system `B'. For example, a plant with N different crystal systems will develop and maintain N bespoke surrogate models that show poor interchangeability despite the presence of significant commonalities in the general crystallization process. To address this issue, we took inspiration from the disruptive emergence of large-language models (LLMs) that leverage powerful transfer learning capabilities of transformers to generalize across different languages or tasks. To this end, a novel time-series-transformer (TST) framework with varying numbers of encoders, decoders, and attention heads was constructed.

Further, observable process data from N (i.e., 10 to 20+) different crystallizers, each with 5000+ operation conditions, was collected. Next, this large corpus of data was utilized to develop a single TST model (i.e., CrystalGPT) to act as a unified surrogate model for any of N systems, and even for a new N+1^th crystal system. Further, CrystalGPT was integrated with a model predictive controller (MPC) to perform setpoint tracking for a batch crystallizer for a chosen crystal system K. Powerful predictive capabilities of CrystalGPT are demonstrated with a combined normalized-mean-squared-error (NMSE) of 5×10^-4 over 10M+ data points, thereby showcasing an 8 to 10 times better performance than current SOTA ML models. Overall, the current work focuses on the development of a plant-wide surrogate model that leverages TST's transfer learning capabilities to seamlessly generalize across [N^th, N+1^th, N+2^th, ...] crystal systems with high-value implications in process monitoring and control.