(16c) A Modernized Approach to Root Cause Analysis By Incorporating Large Language Models, Sentiment Analysis, and Automation

Root Cause Analysis (RCA) is evolving, leveraging advancements in Artificial Intelligence (AI) and Large Language Models (LLMs) to improve efficiency and accuracy. This paper discusses modern techniques for RCA, highlighting how both broad and specialized LLMs can enhance traditional methods, especially in remote settings. These AI applications aim to augment RCA practices and support human collaboration, rather than replace them. Additionally, these approaches enable remote subject matter experts to actively participate in investigations, bringing their expertise without the need for on-site presence. We cover automated polling, AI-driven bias detection, pre-work automation, customizable agendas, and the role of custom LLMs, while addressing challenges like ethics and AI safety. These insights are intended for process improvement professionals and engineers seeking to understand and adopt AI-enhanced RCA methodologies.

Introduction to LLMs in RCA: In the past three years, investments in Large Language Models (LLMs) have surged, enabling a shift in Root Cause Analysis (RCA) practices. These advances have widened the gap between traditional RCA methods and potential improvements powered by AI. The ready availability of general knowledge LLMs, such as OpenAI’s GPT-4o and Anthropic’s Claude 3.5 Sonnet, contrasts with the growing value of custom, domain-specific LLMs built on tailored datasets. Broad LLMs serve as versatile tools with wide-reaching applications but limited depth, whereas custom LLMs provide focused insights aligned with specialized datasets. Both broad and specific LLMs present opportunities to enhance RCA, enabling streamlined workflows, greater accuracy, and improved collaboration.

Broad LLM Applications in RCA: Broad LLMs, accessible through API endpoints, private instances, and open-source options, are cost-effective and versatile for RCA implementation. Four primary applications illustrate their utility. First, automated polling systems digitize team feedback, supporting a streamlined feedback loop. Second, LLMs can detect sentiment and bias within this feedback, drawing on diverse data sources to highlight subjective influences. Third, these insights enable automated pre-work, reducing preparation time by summarizing key issues and themes. Lastly, LLMs facilitate rapid customization of strategies and agendas, leveraging their broad knowledge to craft event-specific frameworks. Together, these applications enhance RCA readiness and agility.

Automated Polling System: Automated polling systems captures team insights in advance of RCA events, encouraging participants to share initial thoughts and observations. This approach highlights preliminary issues and engages the team for productive dialogue. LLMs enhance the value of this process of collecting poll responses by at least 10x, revealing common themes, essential nuances, and significant points for discussion. Using it in this way offers facilitators a preliminary understanding of the team’s perspectives. Summarizing responses enables faster, more digestible pre-work that ensures facilitators are well-prepared. As this technology expands its capabilities, enhanced contextual analysis will allow LLMs to identify knowledge gaps or discrepancies within team feedback, allowing more targeted follow-up questions. Additionally, LLMs could dynamically structure the discussion framework in real time, adjusting to the complexity of initial inputs and fostering quicker alignment and focus on RCA discussions.

AI-Driven Bias Detection: Sentiment analysis is a natural language processing (NLP) method that accepts a given text input and outputs a quantification of polarity [1]. The performance of models aiming to measure sentiment skyrocketed with the introductions of transformers, and LLMs are at the beginning of changing the way we do sentiment analysis again. These models are used to identify cognitive biases, such as confirmation bias or groupthink, within team responses. Detecting these biases early helps ensure RCA efforts remain objective, and evidence based. By analyzing language patterns indicative of bias, LLMs provide facilitators with a summary of potential biases, helping them to navigate discussions with a focus on objectivity. LLMs will progress towards context-sensitive bias detection, identifying biases to particular RCA topics or objectives. The main barrier is that LLMs are less accurate than traditional NLP models specifically designed for sentiment analysis. However, this improvement would allow facilitators to understand and address biases more precisely. Adaptive recommendations from LLMs could also give real-time suggestions, enabling facilitators to address biases immediately, which will lead to enhanced accuracy of root cause findings.

Pre-Work Automation: Pre-work automation utilizes team insights to compile relevant information and draft a structured agenda, allowing facilitators to prepare for RCA sessions efficiently. In layman’s terms, this is handing off the busy work for algorithms to handle, so your teams can focus on the problem at hand. By organizing team inputs, LLMs assemble background data aligned with RCA frameworks, draft preliminary agendas, and highlight information gaps, providing facilitators a comprehensive overview. As LLMs improve they will be able to gather and categorize data from diverse sources, such as historical RCA reports and operational metrics, for even more streamlined pre-work. Real-time agenda optimization based on live inputs will help facilitators adapt session objectives dynamically, improving the precision and relevance of RCA findings.

Customizable Agendas: Once the previous methods have been incorporated into an organization’s RCA process, agendas can be tailored to each RCA event’s unique requirements, ensuring facilitators follow a structured but adaptable approach. Leveraging initial insights and data points, LLMs create a customized agenda based off industry standards for RCA, like building a causal tree, while still allowing facilitators the flexibility to pivot as needed. By simulating expertise in the problem’s specific field, LLMs empower facilitators—often skilled in problem-solving methodologies rather than the field itself—to use these outputs effectively. LLMs will enhance agenda customization by aligning it with team patterns and industry-specific trends, making it more contextually relevant. Responsive agendas that adapt in real time will continuously monitor session dynamics, offering focused and adaptable guidance to the facilitator as new points emerge.

Specialized LLMs for RCA Customization: Using customized models built on proprietary data can modernize RCA by incorporating in-depth knowledge specific to the organization’s processes, personnel, and objectives. There is a significant advantage to this approach considering custom LLMs possess detailed knowledge of business practices and can directly suggest specific improvements to standard operating procedures (SOPs) and workflows. On the same hand, tailored model architectures allow for cost management and open unique use cases, with additional cybersecurity benefits through controlled data usage. However, implementing these models requires a foundational digital transformation, representing a significant project and ongoing adaptation in a rapidly evolving technological landscape.

Barriers and Ethical Considerations: Challenges in adopting AI for RCA include accessibility, cost, and workforce training requirements. AI safety and ethics—such as ensuring unbiased analysis and protecting team member privacy—remain vital considerations. Teams should have the knowledge to question the accuracy of LLM outputs as factors like hallucinations and other forms of undesired outputs can occur. The same methods of Poka-yoke, or mistake-proofing, apply when rolling out these models into new workflows. To safely use LLMs in RCA workflows, it's essential to use secure data practices, ensure clear model outputs, and have ways to manage bias.

Streamlining Safety in Action: Picture a chemical manufacturing plant plagued by recurring production halts due to equipment failures and safety concerns. Traditional RCAs took days to gather data, sift through incident reports, and manage team feedback—often leaving the team grappling with delays and incomplete insights. By integrating AI-driven RCA enhancements, the plant’s safety and engineering teams now swiftly collect and analyze team observations through automated polling, pinpoint potential biases that could skew root cause identification, and structure agendas with tailored pre-work that spotlights high-priority issues, like excessive vibrations or temperature fluctuations. With a specialized LLM trained on plant-specific data, the team even receives targeted recommendations for SOP improvements and equipment maintenance. This modernized RCA process not only speeds up issue resolution but also strengthens safety measures, transforming the plant’s approach to proactive problem-solving.

Conclusion: Advancements in LLMs have created a significant opportunity to modernize RCA, enhancing objectivity, accessibility, and efficiency. Broad LLMs offer flexible, scalable tools for preliminary data gathering and agenda-setting, while specialized models bring customized insights that cater to organizational knowledge and processes. With careful consideration of ethical and technical challenges, AI-driven RCA methodologies can support more accurate, insightful, and collaborative problem-solving, equipping organizations with the tools to achieve a higher standard of root cause identification and resolution.

Works Cited

1. Hugging Face. Sentiment Analysis with Hugging Face Transformers and Python; Hugging Face Blog: https://huggingface.co/blog/sentiment-analysis-python (accessed Nov 3, 2024).