(532g) Integrating Reinforcement Learning into Process Control Curriculum with a Water Level Control Example

A typical chemical engineering process control course covers topics such as dynamic modeling, linearization, stability, Laplace transforms and transfer functions, the response of first and second-order systems, time delay, control strategies (including PID), and controller tuning. In the chemical industry, PID control is widely used to manage various process variables like temperature, pressure, flow rate, or level. Due to the complexity of chemical processes, reinforcement learning is gaining interest in both academia and industry. Instructors are seeking ways to integrate reinforcement learning into chemical engineering process control courses with simple, domain-specific examples.

We developed a Simulink-based water level control example using reinforcement learning that instructors can easily integrate into their curriculum. Simulink is a widely used block-based environment that allows students to model dynamic systems, design control algorithms, and run closed-loop simulations. Many chemical engineering process control courses utilize Simulink.

Our example consists of a Simulink model of a water tank system controlled with a PI controller to maintain the water level at a desired set point. To implement reinforcement learning approach, we created an environment and replaced the PI controller with an Actor-Critic Reinforcement Learning Agent. We used a Deep Deterministic Policy Gradient (DDPG) agent as the actor, trained it, and validated the results. The reinforcement learning approach was effective at maintaining the water level at the set point. With both PI control and reinforcement learning components in this example, instructors can easily build on the existing syllabus and introduce reinforcement learning using a system that students are already familiar with.

To further assist instructors in integrating reinforcement learning into their chemical engineering curriculum, we will provide freely available self-paced online training courses, videos, and project ideas.