Real-time optimization (RTO) and model predictive control (MPC) are essential for enhancing the efficiency and dynamic performance of complex energy systems. However, their effectiveness hinges on accurate and computationally tractable dynamic models. This work addresses this challenge by proposing a transformer neural network architecture specifically designed for modeling energy system dynamics for RTO/MPC applications. The architecture incorporates a modified attention mechanism inspired by positional embedding from vision transformers and task-specific adaptations to the decoder stack. We evaluated the proposed transformer using operational data from a 450 MW coal-fired power plant, a representative large-scale energy system. Compared to conventional recurrent neural networks (GRU and LSTM), the transformer demonstrated superior modeling capability, achieving a 6% increase in R-squared prediction accuracy and an 83% reduction in mean squared error. Critically for real-time application, computation time was reduced by 84%. These results highlight the potential of transformer architectures to provide faster, more accurate dynamic models, enabling improved real-time optimization and control of energy systems.
