Visual navigation is imperative for successful asteroid exploration missions. In this study, an integrated visual navigation system was proposed based on angles-only measurements to robustly and accurately determine the pose of the lander during the final landing phase. The system used the lander's global pose information provided by an orbiter, which was deployed in space in advance, and its relative motion information in adjacent images to jointly estimate its optimal state. First, the landmarks on the asteroid surface and markers on the lander were identified from the images acquired by the orbiter. Subsequently, an angles-only measurement model concerning the landmarks and markers was constructed to estimate the orbiter's position and lander's pose. Subsequently, a method based on the epipolar constraint was proposed to estimate the lander's inter-frame motion. Then, the absolute pose and relative motion of the lander were fused using an extended Kalman filter. Additionally, the observability criterion and covariance of the state error were provided. Finally, synthetic image sequences were generated to validate the proposed navigation system, and numerical results demonstrated its advance in terms of robustness and accuracy.