Lightweight Multiscale Spatio-Temporal Graph Convolutional Network for Skeleton-Based Action Recognition

Using skeletal information to model and recognize human actions is currently a hot research subject in the realm of Human Action Recognition (HAR). Graph Convolutional Networks (GCN) have gained popularity in this discipline due to their capacity to efficiently process graph-structured data. However, it is challenging for current models to handle distant dependencies that commonly exist between human skeleton nodes, which hinders the development of algorithms in related fields. To solve these problems, the Lightweight Multiscale Spatio-Temporal Graph Convolutional Network (LMSTGCN) is proposed. Firstly, the Lightweight Multiscale Spatial Graph Convolutional Network (LMSGCN) is constructed to capture the information in various hierarchies, and multiple inner connections between skeleton joints are captured by dividing the input features into a number of subsets along the channel direction. Secondly, the dilated convolution is incorporated into the temporal convolution to construct Lightweight Multiscale Temporal Convolutional Network (LMTCN), which allows to obtain a wider receptive field while keeping the size of the convolution kernel unchanged. Thirdly, the Spatio-Temporal Location Attention (STLAtt) module is used to identify the most informative joints in the sequence of skeletal information at a specific frame, hence improving the model’s ability to extract features and recognize actions. Finally, multi-stream data fusion input structure is used to enhance the input data and expand the feature information. Experiments on three public datasets illustrate the effectiveness of the proposed network.