The significant illumination variations on the lunar surface pose a substantial challenge to spacecraft landing site localization. Traditional image processing techniques, such as histogram equalization and Retinex-based methods, struggle to adapt to the highly variable lighting conditions prevalent on planetary surfaces. To address the issue of illumination inconsistency between orbital remote sensing images and descent camera images-which often leads to matching failures in vision-based landing site localization pipelines—a novel localization framework that integrates a dedicated illumination control algorithm named LomFormer (Light on Moon) is proposed. Based on a Transformer architecture, LomFormer is designed to actively control the illumination angle in planetary surface images, effectively transforming an image captured under one lighting condition to appear as if it were taken under another. For model training, a multi-angle illumination dataset of the lunar surface was generated by rendering images based on real lunar Digital Elevation Model (DEM) data under various lighting azimuths and elevations. Training and validation results demonstrate that the proposed method achieves promising performance on real data, significantly enhancing the accuracy of lunar image matching and exhibiting strong robustness across diverse illumination scenarios. This research provides a novel perspective on tackling illumination-related challenges in spacecraft landing site localization and introduces a new paradigm for mapping and localization tasks in future autonomous planetary exploration missions.
- Article type
- Year
Stereo vision has emerged as a key technical approach for perceiving 3D terrain in lunar exploration, owing to its advantages such as low cost and high reliability. In recent years, the stereo matching methods based on deep learning have become the mainstream solution for achieving high-precision stereo vision. However, limited by data acquisition conditions, there is currently a lack of public stereo matching datasets tailored to the lunar surface environment. This severely restricts the training and fine-tuning of deep learning models in lunar surface scenarios, thereby impairing their adaptability to complex lunar terrain. To address this issue, this paper proposes Render3D, a self-supervised learning method for lunar surface stereo matching based on 3D rendering. The proposed method requires only monocular panoramic lunar surface images as input. By integrating the high-fidelity surface rendering capability of 2D Gaussian Splatting (2DGS) with the accurate geometric reconstruction capability of Neural Radiance Fields (NeRF), it generates high-quality pseudo-annotated training samples. These samples guide the fine-tuning of the stereo matching model to adapt to the lunar surface environment. Experiments conducted in both simulated lunar surface environment and real physical scenario demonstrate that the stereo matching model fine-tuned using Render3D method significantly outperforms other methods in terms of accuracy. In lunar surface scenarios, the proposed method shows clear superiority over existing self-supervised learning approaches, particularly in complex conditions such as textureless regions and areas with high shadows, where the matching error is reduced by approximately 50% compared to baseline methods, achieving state-of-the-art performance. Experimental results fully demonstrate that Render3D can effectively alleviate the constraint of scarce labeled data for lunar surface thus significantly improving the robustness and generalization ability of stereo matching models in the lunar surface environment.
Open Access
Research Article
Issue
Accurate identification of intracranial hemorrhage (ICH) subtypes on non-contrast CT is crucial for prognosis and treatment but remains challenging due to low contrast and blurred boundaries. This study evaluates the zero-shot performance of multi-modal large language models (MLLMs) versus traditional deep learning in ICH detection and subtyping.
Using 192 NCCT volumes from the RSNA dataset, we compared MLLMs (GPT-4o, Gemini 2.0 Flash, Claude 3.5 Sonnet V2) with deep learning models (ResNet50, Vision Transformer). MLLMs were prompted for ICH presence, subtype, localization, and volume estimation.
Traditional deep learning models outperformed MLLMs in both ICH detection and subtyping. For subtyping, MLLMs showed lower accuracy, with Gemini 2.0 Flash achieving a macro-averaged precision of 0.41 and F1 score of 0.31.
While MLLMs offer enhanced interpretability through language-based interaction, their accuracy in ICH subtyping remains inferior to deep learning networks. Further optimization is needed to improve their utility in three-dimensional medical imaging.
京公网安备11010802044758号