The vehicle routing problem with time windows (VRPTW) is a classical NP-hard combinatorial optimization problem, where NP denotes nondeterministic polynomial time, and it plays a critical role in modern logistics and transportation systems. Although competitive swarm optimization (CSO) algorithms have demonstrated strong performance in continuous optimization, their effective application to discrete combinatorial problems such as VRPTW remains challenging. In this paper, a hybrid competitive swarm optimization and tabu search algorithm (CSO-TS) is proposed to solve the VRPTW. To enhance the search capability of the CSO framework, a tabu search mechanism with an adaptive neighborhood operation strategy is integrated. Moreover, to achieve a better balance between exploration and exploitation, a path diversity index is introduced to quantitatively evaluate solution diversity based on four distinct indices. The proposed CSO-TS algorithm is tested on 56 Solomon benchmark instances and obtains 23 optimal solutions. Extensive computational experiments and comparative analyses demonstrate that CSO-TS outperforms or is competitive with nine state-of-the-art algorithms in terms of solution quality.