TY  - JOUR
AU  - Li, Yinkun
AU  - Zhou, Yawen
AU  - Zhou, Yufeng
AU  - Chai, Li
PY  - 2026
TI  - Adaptive optimal tracking control for underactuated surface vessels using extended state observer and reinforcement learning
JO  - Journal of Automation and Intelligence
SN  - 2097-504X
SP  - 24
EP  - 34
VL  - 5
IS  - 1
AB  - This paper investigates the adaptive optimal tracking control (AOTC) for underactuated surface vessels (USVs). Compared to the majority of existing studies, the control strategy in this paper innovatively combines an extended state observer (ESO) with reinforcement learning (RL). The designed ESO has high estimation accuracy and robust disturbance rejection capabilities for the unmeasurable information for USVs. To obtain the AOTC, the actor–critic (AC) networks based on RL are constructed to solve the Hamilton–Jacobi–Bellman (HJB) equations. Due to the uncertainties, it is challenging to obtain the optimal controller by directly solving the HJB equations. To address this issue, this paper employs neural networks (NNs) to approximate the uncertainties and solves the optimal controller via AC-RL and ESO. In addition, the adaptive parameters of the optimal controller is trained in parallel with AC networks, which can ensure that the trained networks can further improve tracking performance. The boundedness of AOTC for USVs is shown by Lyapunov stability theorem. Finally, simulation results demonstrate the effectiveness of the proposed algorithm.
UR  - https://doi.org/10.1016/j.jai.2025.09.002
DO  - 10.1016/j.jai.2025.09.002