ImVoxelENet: Image to voxels epipolar transformer for multi-view RGB-based 3D object detection

The task of detecting three-dimensional objects using only RGB images presents a considerable challenge within the domain of computer vision. The core issue lies in accurately performing epipolar geometry matching between multiple views to obtain latent geometric priors. Existing methods establish correspondences along epipolar line features in voxel space through various layers of convolution. However, this step often occurs in the later stages of the network, which limits overall performance. To address this challenge, we introduce a novel framework, ImVoxelENet, that integrates a geometric epipolar constraint. We start from the back-projection of pixel-wise features and design an attention mechanism that captures the relationship between forward and backward features along the ray for multiple views. This approach enables the early establishment of geometric correspondences and structural connections between epipolar lines. Using ScanNetV2 as a benchmark, extensive comparative and ablation experiments demonstrate that our proposed network achieves a 1.1% improvement in mAP, highlighting its effectiveness in enhancing 3D object detection performance. Our code is available at https://github.com/xug-coder/ImVoxelENet.