Precise monitoring of perinatal behaviors in ewes is crucial to improve reproductive efficiency with the low risk of dystocia in an intelligent livestock system. However, the current recognition of perinatal behaviors from video data has remained a major challenge, due to the long temporal duration, subtle motion patterns, and high inter-class similarity among behavioral features. In this study, a robust and efficient multi-modal framework was developed to recognize skeleton behavior using an improved Two-Stream Adaptive Graph Convolutional Network (2s-AGCN). A graph deep learning model was designed to capture both spatial and temporal dependencies in complex ewe behaviors. (1) Weighted Adaptive Graph Convolution Layer (W-AGCL) was introduced to overcome the limited adaptability of fixed skeletal topology when representing subtle or dynamic motion. The connection strengths between skeletal nodes were dynamically adjusted to adaptively learn the most informative spatial relationships, according to the behavioral context. The adaptive weighting mechanism enhanced the model sensitivity to spatial structure variations and the robustness to noise and individual differences among ewes. (2) Spatial Temporal Enhanced Attention Module (STE) was developed for the even temporal distribution and the presence of micro-movements in perinatal behaviors. The spatiotemporal regions were selectively emphasized to assign the higher attention weights into frames and joints with discriminative information, thereby improving the network to capture subtle but behaviorally significant cues. Furthermore, a four-stream graph convolutional architecture was proposed for the multi-modal feature fusion. Simultaneously, four complementary modalities were processed, including the Joint Stream representing skeletal joint coordinates, the Bone Stream capturing limb connectivity and orientation, the Joint Motion Stream describing temporal displacement of joints, and the Bone Motion Stream modeling dynamic variations in bone vectors. Among them, feature learning and deep interaction were integrated to clarify the static posture configurations and dynamic motion. A video dataset was constructed for the ewe perinatal behavior, including annotated samples of typical behaviors, such as standing, lying, turning, nest-building, and lambing. Experiments were then conducted to verify the model. The results show that the improved 2s-AGCN model achieved a Top-1 classification accuracy of 86.21%, thus outperforming several skeleton action recognition models, including ST-GCN, ST-GCN++, PoseC3D, Shift-GCN, CTR-GCN, and the original 2s-AGCN. Specifically, the Top-1 accuracies were 7.81, 8.41, 7.95, 7.26, 7.11, and 6.89 percentage points, respectively. The better performance was achieved in the compact architecture with 5.70 million parameters, and the average inference latency per frame was 15.5 ms, fully supporting real-time monitoring in farm environments. The skeleton graph convolutional models were used to recognize the fine-grained animal behaviors in the perinatal period. The improved 2s-AGCN framework effectively balanced accuracy, efficiency, and real-time inference. Spatiotemporal dependencies were adaptively learned to integrate multi-modal skeletal information. The findings can provide a powerful tool for automatic behavior in smart sheep farming. Deep graph learning can be expected to monitor livestock in future applications, such as early lambing prediction, automatic reproduction, and precision welfare assessment in smart pastures.
- Article type
- Year
- Co-author
An accurate and rapid identification of sheep periparturient behaviors can often be required to prevent the potential health risks and production abnormalities in modern animal husbandry. It is crucial to safeguard the ovine welfare for reproductive efficiency, thus reducing neonatal lamb mortality. However, some challenges still remain in the accurate recognition of the behavior during the periparturient period. Particularly, there is a high similarity between the behavioral traits and environmental interferences, such as the complex illumination and cluttered backgrounds in sheep farming. In this study, an advanced recognition was proposed to integrate an enhanced YOLOv8n-Pose key-point model with a backpropagation (BP) neural network. Specifically, an additional P2 detection layer was incorporated into the network architecture, in order to improve the precision of the key-point detection. The fine-grained and small-scale features were captured to accurately localize the anatomical key points in the complex behavioral scenarios. The detection layer was added for a higher degree of spatial resolution. Particularly, there were subtle movement variations in the periparturient sheep. Furthermore, a multi-scale attention block (MAB) module was introduced into the framework, in order to mitigate the feature representation in dynamic environments. A dynamic weighting module was employed to interactively learn the global and local spatial dependencies. Consequently, the robustness and generalization performance of the improved model was achieved under heterogeneous illumination. The MAB module effectively prioritized the most discriminative feature regions, thereby reducing the impact of the background noise and occlusions commonly observed in practical farming environments. The L1-norm channel pruning was systematically implemented to reduce the excessive parameters in the practical deployment constraints. The parameter compression was effectively optimized to eliminate the redundancy in the refined model. An optimal combination was achieved to balance computational efficiency and performance retention. The pruning was utilized to maintain the model integrity using structured sparsity, in order to significantly reduce the computational overhead. The real-time livestock monitoring was realized as suitable for edge computing. A multidimensional dataset of the behavioral feature was constructed to accurately extract the 12 key-point coordinates. Five joint angle parameters were integrated with two pairs of the key-point relative distance metrics, and the key-point detection confidence scores. The dataset was obtained with a 32-dimension feature vector. These feature representations were extracted to serve as the input into a BP neural network for the precise classification of the periparturient behaviors. The BP neural network was trained using adaptive learning. The complex spatiotemporal dependencies were effectively captured among the extracted features. The high classification accuracy was achieved after extraction. A series of experiments were conducted to evaluate the performance of the improved model on a self-developed dataset of periparturient sheep. The results demonstrated that the improved YOLOv8n-Pose model achieved a notable 4.6 percentage point increase in the mean average precision (mAP50) and a 6.7 percentage point improvement in mAP50:95 for the key-point detection, compared with the baseline architecture. Moreover, the BP neural network exhibited outstanding performance in the classification. An F1-score of 95.7% was obtained to distinguish the critical periparturient behaviors. The superior efficacy of the key-point recognition was obtained to identify the periparturient behavior. Ultimately, the robust technical framework greatly contributed to the intelligent livestock systems. Full automation and precision monitoring were enhanced in sheep farming.
京公网安备11010802044758号