Leveraging multi-modal and historical knowledge graphs for continual robot navigation

In contemporary robotic navigation systems, autonomous agents are increasingly required to operate in complex, dynamically evolving environments. However, in dynamic environments, the disparity between heterogeneous sensor inputs and continuously changing conditions frequently induces catastrophic forgetting in learned models. To address these challenges, we propose an integration of multi-modal graph with historical knowledge graph for continual robotic navigation. This integration combines the structured multi-modal knowledge graph (SMKG) module and the historical dynamic knowledge graph (HDKG) module to enable dynamic multi-modal representation updating while preventing catastrophic forgetting through selective preservation of critical graph components. Specifically, to effectively integrate multi-modal information from visual, light detection and ranging (LiDAR), and odometry data in navigation systems, an SMKG is constructed that unifies these heterogeneous sensory inputs into a relational knowledge representation, where nodes represent samples and edges encode their semantic-spatial relationships. Furthermore, we propose an HDKG that preserves past knowledge and integrates task semantics, dynamically adapting its structure to maintain context for novel tasks. The proposed method preserves and dynamically integrates the multi-modal knowledge with historical task information through contextually rich representations that are continuously updated through aggregation of the historical knowledge graph during ongoing learning. Experimental results validate the effectiveness of the proposed method in evaluating the multi-modal continual learning performance of robotic navigation across sequential environments.