Masked autoencoders (MAEs) have emerged as a powerful self-supervised approach for point cloud analysis. Nevertheless, existing methods often separately focus on global structures or multi-scale features, ignoring their complementary potential. In this paper, we propose a novel dual-Siamese masked autoencoder (DS-MAE) framework that explores integrating global and hierarchical feature learning in a unified architecture for point cloud analysis. In particular, we introduce a consistent dual-branch patch embedding strategy to partition the point cloud into patches using shared group centers, ensuring both global and hierarchical branches process point patches centered at the same spatial locations. Each branch employs dual-branch Siamese encoders to process original and augmented point patches, learning representations that capture both local details and global context. In addition, we have designed cross-attention Siamese decoders to reconstruct masked point patches and align features both within and between branches with crossattention mechanisms. Comprehensive experiments demonstrate our method consistently achieves superior results to prior methods. Code is available at https://github.com/shaoandy1211/DS-MAE.git.
- Article type
- Year
- Co-author
Open Access
Research Article
Issue
Graph convolutional networks (GCNs) have become a dominant approach for skeleton-based action recognition tasks. Although GCNs have made significant progress in modeling skeletons as spatial-temporal graphs, they often require stacking multiple graph convolution layers to effectively capture long-distance relationships among nodes. This stacking not only increases computational burdens but also raises the risk of over-smoothing, which can lead to the neglect of crucial local action features. To address this issue, we propose a novel multi-scale adaptive large kernel graph convolutional network (MSLK-GCN) to effectively aggregate local and global spatio-temporal correlations while maintaining the computational efficiency. The core components of the network include two multi-scale large kernel graph convolution (LKGC) modules, a multi-channel adaptive graph convolution (MAGC) module, and a multi-scale temporal self-attention convolution (MSTC) module. The LKGC module adaptively focuses on active motion regions by utilizing a large convolution kernel and a gating mechanism, effectively capturing long-distance dependencies within the skeleton sequence. Meanwhile, the MAGC module dynamically learns relationships between different joints by adjusting connection weights between nodes. To further enhance the ability to capture temporal dynamics, the MSTC module effectively aggregates the temporal information by integrating Efficient Channel Attention (ECA) with multi-scale convolution. In addition, we use a multi-stream fusion strategy to make full use of different modal skeleton data, including bone, joint, joint motion, and bone motion. Exhaustive experiments on three scale-varying datasets, i.e., NTU-60, NTU-120, and NW-UCLA, demonstrate that our MSLK-GCN can achieve state-of-the-art performance with fewer parameters.
京公网安备11010802044758号