In organic chemistry, macroscopic visual cues—such as solid formation in crystallization, liquid level changes in distillation, and color shifts in titration—are crucial for decision-making. However, current automation systems lack visual perception, requiring human oversight. HeinSight is a computer vision system that monitors these cues in real-time and enables automated experimental control based on visual feedback (e.g., cooling a reactor until nucleation occurs). Initially, HeinSight was integrated with EasyMax, a widely used automated lab reactor, where it worked alongside iControl, the reactor’s commercial software. HeinSight processed visual data in Python and relayed it to iControl, which adjusted process parameters accordingly. However, iControl posed two key limitations: (1) restricted experimental design, as it only supports predefined functions, and (2) limited hardware integration, as it exclusively controls Mettler Toledo devices. To overcome these challenges, we propose replacing iControl with ivoryOS, an open-source user interface for Self-Driving Laboratories (SDLs). In this talk, we will demonstrate how this transition enables:
1. Greater workflow adaptability by integrating external hardware for broader experimental control (e.g., solid dosing units, HPLC, and external pumps).
2. Multi-camera synchronization to coordinate multi-reactor setups, demonstrated in continuous stirred-tank reactors.
3. A unified architecture for scalability beyond EasyMax, allowing seamless integration with larger jacketed reactors for industrial applications.
By merging computer vision with flexible experimental scripting, this work advances automated chemistry, paving the way for more autonomous and scalable self-driving labs.
