Vehicle state estimation accuracy in the Adaptive Unscented Kalman Filter (AUKF) is highly dependent on the proper tuning of the process and measurement noise covariance matrices. Conventional optimization techniques often produce suboptimal covariance values, increasing the risk of estimation divergence and compromising vehicle stability. To overcome this limitation, this paper proposes a Modified Dragonfly Optimization Algorithm-based Adaptive Unscented Kalman Filter (MDOA-AUKF), in which adaptive covariance tuning is formulated as an optimization problem. By incorporating elite solution preservation and multi-behavioral swarm dynamics, the proposed MDOA effectively balances exploration and exploitation, thereby enabling adaptive updates of the process (Q) and measurement (R) noise covariance matrices at each iteration. This improves the convergence, estimation accuracy, and robustness under nonlinear driving conditions and varying vehicle mass. The proposed approach is implemented using a two-degree-of-freedom bicycle model considering lateral and yaw dynamics under a low-friction road condition (tire–road friction coefficient of 0.4) and evaluated using Fishhook (FH) and Double Lane Change (DLC) maneuvers. During the FH maneuver, the proposed method achieves root mean squared error (RMSE)/ normalized root mean squared error (NRMSE) values of 0.0008/0.21% and 0.0010/0.03% for the sideslip angle and yaw rate, respectively, while the corresponding values during the DLC maneuver are 0.0011/0.24% and 0.0009/0.02%, respectively. Compared with Particle Swarm Optimization (PSO), Ant Colony Optimization (ACO), and Grey Wolf Optimization (GWO), the proposed approach provides significantly lower estimation errors and faster convergence. Robustness is further validated under vehicle mass variations from 1800 kg to 2300 kg, demonstrating consistently low estimation errors. These results confirm that the proposed MDOA-AUKF is an accurate, robust, and computationally efficient framework for intelligent vehicle state estimation.