A Hybrid Deep Learning Approach for Real-Time Cheating Behaviour Detection in Online Exams Using Video Captured Analysis

Online examinations have become a dominant assessment mode, increasing concerns over academic integrity. To address the critical challenge of detecting cheating behaviours, this study proposes a hybrid deep learning approach that combines visual detection and temporal behaviour classification. The methodology utilises object detection models—You Only Look Once (YOLOv12), Faster Region-based Convolutional Neural Network (RCNN), and Single Shot Detector (SSD) MobileNet—integrated with classification models such as Convolutional Neural Networks (CNN), Bidirectional Gated Recurrent Unit (Bi-GRU), and CNN-LSTM (Long Short-Term Memory). Two distinct datasets were used: the Online Exam Proctoring (EOP) dataset from Michigan State University and the School of Computer Science, Duy Tan Unievrsity (SCS-DTU) dataset collected in a controlled classroom setting. A diverse set of cheating behaviours, including book usage, unauthorised interaction, internet access, and mobile phone use, was categorised. Comprehensive experiments evaluated the models based on accuracy, precision, recall, training time, inference speed, and memory usage. We evaluate nine detector–classifier pairings under a unified budget and score them via a calibrated harmonic mean of detection and classification accuracies, enabling deployment-oriented selection under latency and memory constraints. Macro-Precision/Recall/F1 and Receiver Operating Characteristic-Area Under the Curve (ROC-AUC) are reported for the top configurations, revealing consistent advantages of object-centric pipelines for fine-grained cheating cues. The highest overall score is achieved by YOLOv12 + CNN (97.15% accuracy), while SSD-MobileNet + CNN provides the best speed–efficiency trade-off for edge devices. This research provides valuable insights into selecting and deploying appropriate deep learning models for maintaining exam integrity under varying resource constraints.