(109f) Identifying Key State Variables for AI Control Models Using Causal Effects

1. Introduction

In recent years, artificial intelligence (AI)-driven control systems have gained significant attention as industries explore the potential of reinforcement learning (RL) to automate plant operations. Model-based offline RL has emerged as a promising approach in industrial settings where safety concerns make trial-and-error operations infeasible. By leveraging historical operation data, this method builds an AI control model that learns plant dynamics and optimizes control actions to achieve target states. This approach has demonstrated success in real-world industrial applications [1, 2].

A critical factor in successful AI control model develpment is the selection of state variables, which form the foundation for decision-making and optimization. In model-based offline RL, the choice of state variables is especially crucial as it directly impacts the accuracy of learned plant dynamics and the effectiveness of control policy.

Traditionally, state variable selection has been performed manually by domain experts. While effective, this process is labor-intensive and requires extensive expertise in both plant processes and control systems. As industries move toward data-driven solutions, numerous machine learning-based approaches have been proposed for variable selection [3]. However, these methods primarily focus on selecting explanatory variables for prediction tasks and face challenges in selecting state variables tailored to plant-specific AI control models. These challenges include:

1. Spurious Correlations: In plants where many variables are governed by controllers, correlations between variables may not reflect causal relationships. Relying on such spurious correlations can lead to suboptimal or erroneous control models.
2. Lack of Interpretability: Dimensionality reduction techniques, such as principal component analysis or autoencoders, implicitly select variables as latent representations. However, this often obscures the rationale behind variable selection, reducing interpretability—an essential requirement for high-stakes industrial applications.
3. Mismatch Between Prediction and Control: Models optimized for state prediction may not be ideal for control tasks. For instance, while dynamics models focus on predicting state transitions, they may neglect variables critical for control optimization.

To address these challenges, this research proposes a novel method for automatic state variable selection tailored to AI control models. By quantifying causal relationships between variables, the method systematically identifies key state variables critical for control tasks. This approach aims to reduce reliance on expert knowledge while improving the efficiency and accuracy of AI-driven control system development.

2. Methodology

The proposed method leverages plant operation data and piping and instrumentation diagram (P&ID) information to quantify causal relationships between variables and explicitly select state variables relevant to control tasks. The method consists of the following steps:

Step 1: Causal Graph Construction

The first step involves constructing a causal graph that qualitatively represents relationships between variables. This causal graph is derived from the plant’s P&ID, which provides the physical and logical connections of equipment and instrumentation. To capture the dynamics of plant behavior, such as time delays, a time-series causal graph [4] is adopted.

Step 2: Quantification of Causal Effects

Once the causal graph is established, causal effects are quantified using operation data. Causal effects measure how changes in one variable causally influence other variables and are estimated by properly adjusting for confounding variables [5]. To identify control-relevant variables, the method quantifies two types of causal effects for all variables:

1. Effects on the control target: How a variable influences the control target variable.
2. Effects from the control action: How a variable is influenced by the control action variable.

Step 3: State Variable Selection

Using the quantified causal effects, the method selects key state variables critical for the AI control model. Specifically, variables are selected if their effects on the target are greater than their effects from the action. This criterion ensures that the selected variables significantly influence the control target but are not directly manipulable through control actions. These variables are essential for accurately modeling the plant’s dynamics and accounting for external, uncontrollable factors.

Step 4: Validation

Domain experts validate the selected variables to ensure their relevance and effectiveness in improving the performance of the AI control model. This step is crucial to confirm the applicability of the method to real-world plant operations.

By following this approach, the proposed method achieves the following objectives:

1. Avoiding spurious correlations: By explicitly distinguishing correlation from causation, the method ensures that only causally relevant variables are selected.
2. Improved interpretability: Explicitly selecting variables (rather than compressing them into latent representations) enhances the interpretability of the selection process.
3. Control-oriented selection: The method prioritizes variables critical for control optimization over those merely useful for state prediction.

3. Case Study

The proposed method was evaluated using a simulated batch polymerization process for polyvinyl chloride (PVC). In this process, an initiator is added to a reactor containing the raw monomer to start the reaction. As the reaction begins, the reaction heat causes a rapid temperature rise, which the PID controller fails to manage within the desired range. To address this issue, an AI control model was developed to manipulate the PID controller’s setpoint variable (SV). Given the complex interactions between variables—such as reactor temperature, initiator flow rate, and cooling water flow rate—selecting appropriate state variables was critical for building an effective AI control model.

Simulation Setup

• Simulator: Visual Modeler® (Omega Simulation Co., Ltd)
• Data Collection Period: 10 minutes before reaction initiation and 1 hour after the reaction began.
• Operation Conditions: The PID controller’s SV was varied across 73 patterns to simulate diverse operating conditions.
• Sampling Time: Both simulation and control cycles were set to 1 second.
• State Variable Candidates: 21 variables were considered as candidates.

Evaluation Procedure

1. Variable Selection Validation: Key variables identified by the proposed method were compared with those deemed important by domain experts to assess selection accuracy.
2. Control Model Performance Evaluation: An AI control model was trained using the selected state variables, with process dynamics modeled as a Markov decision process. State variable combinations and discretization levels were optimized using information entropy. A model-based offline RL method, policy iteration, was used to derive the control policy. Control performance was measured by the average absolute deviation of the reactor temperature from the target range of 67±0.5°C.

4. Results and Discussion

The case study results demonstrate the effectiveness of the proposed method in accurately selecting key state variables and improving control performance in a complex batch polymerization process.

Key Findings

1. Variable Selection Accuracy: The proposed method identified 10 key variables from 21 candidates. The selected variables, such as reactor temperature, reactor pressure, and initiator flow rate, aligned perfectly with those identified by domain experts, achieving a 100% agreement. This validates the method’s ability to automate variable selection with expert-level reliability.
2. Control Performance: The AI control model achieved excellent performance, with average temperature deviation of 0.025°C from the target range, significantly better than the PID controller without manual intervention (0.30°C), highlighting the suitability of the selected variables for optimizing control policies.

Limitations

• Dependence on Expert Knowledge for Causal Graph Construction: Initial causal graph construction relies on domain expertise based on P&IDs. Automating this step remains a challenge for future research.
• Simulation-Based Validation: The evaluation was conducted using a simulator, which may not fully capture the complexities of real-world plant operations. Further validation is required to assess robustness under practical conditions, including noise and incomplete data.

Practical Implications

The proposed method provides a systematic and data-driven approach for state variable selection, reducing time and effort required for AI control model development. By focusing on causally relevant variables, the method enhances both the interpretability and performance of AI control models, making it a valuable tool for industrial applications.

5. Conclusions

This research introduced a novel method for state variable selection of AI control models for plant automation. By leveraging causal inference, the method addresses key challenges, including spurious correlations, lack of interpretability, and the mismatch between prediction and control tasks.

Key Contributions

1. Demonstrated the effectiveness of causal inference for identifying critical variables in control tasks.
2. Validated the method through a case study involving a PVC polymerization reactor simulator.
3. Highlighted the potential for reduced development time and improved model performance.

Future Work

1. Automating causal graph construction to further reduce reliance on expert knowledge.
2. Extending the method to real-world plant operations to evaluate its robustness under practical conditions.

By addressing these challenges, the proposed method has the potential to revolutionize AI-driven control system development, paving the way for fully autonomous plant operations.

References

[1] T. Yoshida et al., 2022, “An artificial-intelligent-based realtime control of the steam temperature for an energy-from-waste plant: Demonstration operation results,” Proceedings of the Annual Conference of Japan Society of Material Cycles and Waste Management.

[2] M. Hirano et al., 2024, “A Transparent AI-based Approach for Controlling Processes with Time Delays,” Transactions of the Japanese Society for Artificial Intelligence.

[3] J. Li et al., 2017, “Feature Selection: A Data Perspective,” ACM Computing Surveys.

[4] J. Runge et al., 2023, “Causal Inference for Time Series,” Nature Review Earth & Environment.

[5] J. Runge, 2021, “Necessary and Sufficient Graphical Conditions for Optimal Adjustment Sets in Causal Graphical Models with Hidden Variables,” Advances in Neural Information Processing Systems.