Sort:
Open Access Issue
Teaching Programming System of Virtual Industrial Robot Based on HTC Vive
Journal of Guangdong University of Technology 2023, 40(2): 30-38
Published: 01 March 2023
Abstract PDF (3.6 MB) Collect
Downloads:2

China has become the largest consumer of industrial robots in the world, but the gap of related technical talents is expanding year by year. The important reason is the high cost and dangerous operation of industrial robots, which leads to difficulties in teaching and programming of robots. To solve this problem, based on HTC Vive (a VR device jointly developed by HTC and VALVE), a virtual industrial robot teaching programming system is developed by combining virtual reality (VR) technology with robot teaching programming. First, the VR environment characteristics of the system is analyzed, and a three-dimensional UI (User Interface) designed for the main interface interaction of the system. Then the kinematic modeling of the robot is carried out to realize the teaching method of dragging the robot directly. Using lexical analysis, syntax analysis and sentence execution mechanism, the virtual interpreter is developed, and the program editing mode of inserting instructions quickly is realized. Finally, the system realizes the robot instruction programming function which can carry out complex trajectory motion. Experimental tests show that the system has good interactivity and can be used for virtual robot teaching and programming. The system is safe to operate, and can reduce the cost of robot teaching and programming, contribute to the training of robot related talents, provide solutions for robot simulation in industrial production, and help the development of China’s manufacturing industry.

Open Access Issue
Design of a Coronary Heart Disease Education System Based on AR Technology
Journal of Guangdong University of Technology 2026, 43(1): 49-60
Published: 06 September 2025
Abstract PDF (3.2 MB) Collect
Downloads:1

Coronary heart disease (CHD) , as a complex multifactorial condition, presents significant challenges for patient education due to the abstract and specialized nature of its pathological mechanisms. Conventional health education models relying primarily on static media such as pamphlets and instructional videos exhibit notable limitations in knowledge dissemination efficacy and patient engagement. To address these issues, this study innovatively develops an augmented reality (AR) -based interactive CHD health education system. The system integrates multimodal hardware including HoloLens2, Kinect DK, and Pico4 to establish a HoloLens2-Kinect DK-Pico4 multi-terminal collaborative framework, enabling shared physician-patient AR perspectives. Clinicians overlay real-time cardiovascular models onto patients’ anatomies to simulate disease progression, while patients synchronously observe personalized pathological evolution through headsets, thereby enhancing situational cognition. At the technical implementation level: Personalized educational resources undergo parameterization to optimize multimedia visual presentation; A novel depth-camera registration maintenance algorithm utilizing spatial anchors is proposed, coupled with pose update frequency optimization to significantly enhance cardiovascular model registration stability; Real-time skeletal data captured by Kinect DK is mapped to Unity’s Avatar module, ensuring stable synchronization of cardiovascular models with patient movement; Cross-platform applications (PC server, clinician AR terminal, patient VR terminal) are developed via Unity3D, with seamless HoloLens2-Kinect DK-Pico4 communication achieved through custom data packet structures, message types, and process-pool mechanisms.

Open Access Issue
Method for Data Augmentation of Workpiece Defect Samples Based on Generative Sample Synthesis
Journal of Guangdong University of Technology 2025, 42(3): 27-35
Published: 08 January 2025
Abstract PDF (3.7 MB) Collect
Downloads:4

To address the problem of severe lack of defect data in workpieces to train the deep-learning-based defect visual detection systems, this paper introduces a generative sample synthesis method that integrates generative adversarial networks (GAN) with a physical-based rendering (PBR) pipeline for data augmentation. The method employs ConSinGAN as the defect feature generation model and enhances the discriminator by incorporating a coordinate attention (CA) mechanism, enabling more precise identification of defect features in images. Additionally, the loss function is adjusted by introducing a weighted combination of reconstruction loss and multi-scale structural similarity loss to alleviate the gradient vanishing in small sample training and improve the quality of generated samples. The PBR pipeline is used to output the augmented samples, which first constructs a 3D model for the workpiece to be augmented, and then use poisson blending to merge the generated defect features with the original model texture. Finally, defect samples of the workpiece are rendered in a simulated production environment using a virtual camera. Experimental results on public datasets demonstrate the effectiveness of the proposed method in augmenting small samples of workpiece defects.

Issue
Design of an AR visualization system for industrial robot teaching and training platform
Experimental Technology and Management 2024, 41(4): 118-124
Published: 20 April 2024
Abstract PDF (3.4 MB) Collect
Downloads:4
[Objective]

Industrial robot teaching and training platforms are complex systems comprising numerous devices working in tandem. Given the intricate nature of these platforms and the limited training time available, students often struggle to fully comprehend their operating mechanisms and efficiently complete practical training tasks.

[Methods]

To address these challenges, a multilevel architecture is proposed to design an AR visualization application for the robot training platform. This approach decouples the deeply bound physical equipment, multisource data, construction process, and application performance. It enhances the flexibility of the visualization scheme, making it more versatile and systematically displaying the AR visualization scheme of the robot training platform. The platform’s AR visualization application system is developed using the Unity3D engine. KEPServerEX operates as the OPU CA server to obtain multisource device data and transfer it to the SQL database. The device data is then synchronized with the AR visualization application through SocketAsyncEventArgs. Conventional training guidance construction requires a manual compilation of relevant training task guidance information. To streamline this, we construct training process guidance using a bidirectional sequence operation behavior method. A bidirectional sequence training process directed graph represents connections between different training tasks. Each operation guidance node contains the ID of the operation task, operation content, and prompt label. This structure enables quick generation of the robot training platform status and the official manual information. The coordinates of the robot training platform in the world coordinate system are converted to the HoloLens2 camera coordinate system via image-based registration. This conversion is then extended from the HoloLens2 camera coordinate system to the cropping space through space clipping. Finally, the virtual model is accurately presented in the HoloLens2 binocular picture through the UV pixel space transformation. However, image-based registration may lead to registration loss or drift. To mitigate this, we employ space anchoring technology to achieve tracking registration of the platform model. This approach anchors the virtual model in the world coordinate system, preventing registration drift and loss. The premodeling method ensures the correct occlusion relationship in the virtual–real fusion, maintaining geometric consistency between the virtual model and the real environment.

[Results]

On the HoloLens2 device, the AR visualization presents the structure information of the robot training platform, the operating principle of the device, the status of the electronic control nodes, wiring paths, robot teaching paths, the operating status of the platform, and training operation guidance, Students can gain a comprehensive understanding of the overall operating status of the robot training platform and the key training guidance through multiple means such as gestures, viewpoints, and language.

[Conclusions]

The practice shows that the AR visualization application of the industrial robot teaching and training platform enables students to quickly familiarize themselves with the operating mechanism of the training platform. It offers a systematic understanding of the platform’s actual operating status and facilitates the safe and efficient completion of training tasks.

Total 4