Design of an AR visualization system for industrial robot teaching and training platform

Industrial robot teaching and training platforms are complex systems comprising numerous devices working in tandem. Given the intricate nature of these platforms and the limited training time available, students often struggle to fully comprehend their operating mechanisms and efficiently complete practical training tasks.

To address these challenges, a multilevel architecture is proposed to design an AR visualization application for the robot training platform. This approach decouples the deeply bound physical equipment, multisource data, construction process, and application performance. It enhances the flexibility of the visualization scheme, making it more versatile and systematically displaying the AR visualization scheme of the robot training platform. The platform’s AR visualization application system is developed using the Unity3D engine. KEPServerEX operates as the OPU CA server to obtain multisource device data and transfer it to the SQL database. The device data is then synchronized with the AR visualization application through SocketAsyncEventArgs. Conventional training guidance construction requires a manual compilation of relevant training task guidance information. To streamline this, we construct training process guidance using a bidirectional sequence operation behavior method. A bidirectional sequence training process directed graph represents connections between different training tasks. Each operation guidance node contains the ID of the operation task, operation content, and prompt label. This structure enables quick generation of the robot training platform status and the official manual information. The coordinates of the robot training platform in the world coordinate system are converted to the HoloLens2 camera coordinate system via image-based registration. This conversion is then extended from the HoloLens2 camera coordinate system to the cropping space through space clipping. Finally, the virtual model is accurately presented in the HoloLens2 binocular picture through the UV pixel space transformation. However, image-based registration may lead to registration loss or drift. To mitigate this, we employ space anchoring technology to achieve tracking registration of the platform model. This approach anchors the virtual model in the world coordinate system, preventing registration drift and loss. The premodeling method ensures the correct occlusion relationship in the virtual–real fusion, maintaining geometric consistency between the virtual model and the real environment.

On the HoloLens2 device, the AR visualization presents the structure information of the robot training platform, the operating principle of the device, the status of the electronic control nodes, wiring paths, robot teaching paths, the operating status of the platform, and training operation guidance, Students can gain a comprehensive understanding of the overall operating status of the robot training platform and the key training guidance through multiple means such as gestures, viewpoints, and language.

The practice shows that the AR visualization application of the industrial robot teaching and training platform enables students to quickly familiarize themselves with the operating mechanism of the training platform. It offers a systematic understanding of the platform’s actual operating status and facilitates the safe and efficient completion of training tasks.