To address the issue of unbalanced task distribution between electric bus vehicles and drivers in loop line, this study proposed a joint optimal scheduling model, which mainly improves the overall utilization rate by adjusting vehicles and drivers in clockwise and counterclockwise directions. Given a fixed loop route and non-fixed vehicle-driver assignments, the model considers various constraints such as vehicle mileage, workload, number of charging stations, charging duration, driver working and rest times. It aims to minimize both the total operating cost of the transit enterprise and the total timetable adjustment, while formulating an orderly charging management plan and scheduling strategy for vehicles and drivers. In the aspect of solution, the mixed integer nonlinear programming model was transformed into linear programming model by linear transformation, and the scheduling scheme was obtained by using CPLEX solver. Additionally, a multi-objective particle swarm algorithm (MOPSO) and improved multi-objective particle swarm algorithm (ε-MOPSO) based on constraint processing mechanism were used to solve the scheduling scheme respectively, and the convergence and uniformity of external file set were ensured by grid method. The proposed approach is validated through a case study on Beijing’s Route 200 (inner and outer loop lines). A comparative analysis of the results obtained from the CPLEX solver, the traditional MOPSO, and the improved ε-MOPSO confirms the effectiveness of the improved algorithm. The optimized scheduling plan reduces the number of vehicles from 28 to 23 (a 17.86% reduction) and the number of drivers from 28 to 25 (a 10.71% reduction), thereby lowering the total operating cost. The timetable adjustments average 4.13 minutes per departure, resulting in more evenly spaced departures and better meeting passenger demand. This significantly enhances the operational efficiency of public transportation and holds substantial practical significance.