Satellite orbit decision-making requires a reliable approach in uncertain environments. Classical guidance methods can produce high-quality trajectories but rely on repeated numerical solutions of boundary-value problems, which limits their scalability for large-scale online decision-making. Deep reinforcement learning (RL) can in principle handle nonlinear high-dimensional dynamics, yet its interaction demand makes sample efficiency a central bottleneck for orbital decision-making under high-fidelity astrodynamics models. This paper proposes an RL framework for learning-efficient orbit decision based on an astrodynamics-informed parameterized action space. Each action consists of a discrete maneuver index and a continuous parameter vector. The discrete index selects a maneuver primitive from a library of classical orbital guidance methods. The continuous parameters unify the internal degrees of freedom of these primitives. This architecture with reparameterized continuous sampling and projection enforces maneuver feasibility and generates bounded physically meaningful commands. Comprehensive simulations demonstrate that the proposed astrodynamics-informed parameterization improves sample efficiency and final performance compared with unstructured
- Article type
- Year
- Co-author
Open Access
Issue
The satellite orbital pursuit game focuses on studying spacecraft maneuvering strategies in space. Traditional numerical methods often face real-time inadequacies and adaptability limitations when dealing with highly nonlinear problems. With the advancement of Deep Reinforcement Learning (DRL) technology, continuous-time orbital control capabilities have significantly improved. Despite this, the existing DRL technologies still need adjustments in action delay and discretization structure to better adapt to practical application scenarios. Combining continuous learning and model planning demonstrates the adaptability of these methods in continuous-time decision problems. Additionally, to more effectively handle action delay issues, a new scheduled action execution technique has been developed. This technique optimizes action execution timing through real-time policy adjustments, thus adapting to the dynamic changes in the orbital environment. A Hierarchical Reinforcement Learning (HRL) strategy was also adopted to simplify the decision-making process for long-distance pursuit tasks by setting phased subgoals to gradually approach the target. The effectiveness of the proposed strategy in practical satellite pursuit scenarios has been verified through simulations of two different tasks.
Open Access
Cover Article
Issue
In recent years, the availability of space orbital resources has been declining, and the increasing frequency of spacecraft close approach events has heightened the urgency for enhanced space security measures. This paper establishes a comprehensive framework for intelligent orbital game technology in space, encompassing four core technologies: threat perception of non-cooperative targets, intent recognition, situation assessment, and intelligent orbital game countermeasures. The concepts of multi-turn, multi-round and multi-match in space orbital games are defined, clarifying the core technological requirements for intelligent space orbital games and establishing a cohesive technological framework. Subsequently, the current status of research on these four core technologies is investigated. The challenges faced in the existing research are analyzed, and potential solutions for future studies are proposed. This paper aims to provide readers with a thorough understanding of the latest advancements in space intelligent orbital game technology, along with insights into the future directions and challenges in this field.
Open Access
Full Length Article
Issue
The problem of collision avoidance for non-cooperative targets has received significant attention from researchers in recent years. Non-cooperative targets exhibit uncertain states and unpredictable behaviors, making collision avoidance significantly more challenging than that for space debris. Much existing research focuses on the continuous thrust model, whereas the impulsive maneuver model is more appropriate for long-duration and long-distance avoidance missions. Additionally, it is important to minimize the impact on the original mission while avoiding non-cooperative targets. On the other hand, the existing avoidance algorithms are computationally complex and time-consuming especially with the limited computing capability of the on-board computer, posing challenges for practical engineering applications. To conquer these difficulties, this paper makes the following key contributions: (A) a turn-based (sequential decision-making) limited-area impulsive collision avoidance model considering the time delay of precision orbit determination is established for the first time; (B) a novel Selection Probability Learning Adaptive Search-depth Search Tree (SPL-ASST) algorithm is proposed for non-cooperative target avoidance, which improves the decision-making efficiency by introducing an adaptive-search-depth mechanism and a neural network into the traditional Monte Carlo Tree Search (MCTS). Numerical simulations confirm the effectiveness and efficiency of the proposed method.
京公网安备11010802044758号