In this study, a real-time optimal control approach is proposed using an interactive deep reinforcement learning algorithm for the Moon fuel-optimal landing problem. Considering the remote communication restrictions and environmental uncertainties, advanced landing control techniques are demanded to meet the high requirements of real-time performance and autonomy in the Moon landing missions. Deep reinforcement learning (DRL) algorithms have been recently developed for real-time optimal control but suffer the obstacles of slow convergence and difficult reward function design. To address these problems, a DRL algorithm is developed using an actor-indirect method architecture to achieve the optimal control of the Moon landing mission. In this DRL algorithm, an indirect method is employed to generate the optimal control actions for the deep neural network (DNN) learning, while the trained DNNs provide good initial guesses for the indirect method to promote the efficiency of training data generation. Through sufficient learning of the state-action relationship, the trained DNNs can approximate the optimal actions and steer the spacecraft to the target in real time. Additionally, a nonlinear feedback controller is developed to improve the terminal landing accuracy. Numerical simulations are given to verify the effectiveness of the proposed DRL algorithm and demonstrate the performance of the developed optimal landing controller.

This paper presents the crucial method for obtaining our team’s results in the 8th Global Trajectory Optimization Competition (GTOC8). Because the positions and velocities of spacecraft cannot be completely determined by one observation on one radio source, the branch and bound method for sequence optimization of multi-asteroid exploration cannot be directly applied here. To overcome this difficulty, an optimization method for searching the observing sequence based on nominal low-thrust trajectories of the symmetric observing configuration is proposed. With the symmetric observing configuration, the normal vector of the triangle plane formed by the three spacecraft rotates in the ecliptic plane periodically and approximately points to the radio sources which are close to the ecliptic plane. All possible observing opportunities are selected and ranked according to the nominal trajectories designed by the symmetric observing configuration. First, the branch and bound method is employed to find the optimal sequence of the radio source with thrice observations. Second, this method is also used to find the optimal sequence of the left radio sources. The nominal trajectories are then corrected for accurate observations. The performance index of our result is 128,286,317.0 km which ranks the second place in GTOC8.