Five-phase permanent-magnet synchronous motors (PMSMs) offer high fault tolerance, low torque ripple, and high torque density. An open-circuit fault results in an asymmetric motor model, which causes direct current (DC) bias and harmonics, thus affecting operational performance with sensorless control under fault-tolerant conditions. To improve operational performance, this study proposes a novel sliding mode observer (SMO) based on an enhanced adaptive complex-coefficient filter (EACCF) for the sensorless vector-control strategy of a five-phase PMSM with an open-circuit fault. The novelty of the proposed strategy is the development of the EACCF, which is characterized by an effective DC bias and harmonic-attenuation capability. Additionally, the fundamental back electromotive force can be estimated without phase lag or amplitude attenuation using the SMO. By incorporating a phase-locked loop, the accuracy of position estimation can be improved under both healthy and open-circuit fault conditions. The sensorless control can not only restrain the fluctuating torque caused by open-circuit faults but also offers good steady-state and dynamic performances under healthy and open-circuit fault conditions. Experimental results are presented to demonstrate the feasibility of the proposed method.