A safe control scheme integrating a parameter adaptation approach, distributed extended state observer and reciprocal control barrier function is proposed to address bearing formation control and inter-agent collision safety issues of quadrotor UAV swarms under unmodeled dynamics and external disturbances. Under the leader-follower architecture, a state-disturbance estimation framework is constructed, which only adopts inter-UAV bearing angles, bearing angular rates and relative distance information to online estimate the global position, velocity as well as internal and external disturbances of each follower. A Control Lyapunov Function (CLF) is formulated according to estimated states and desired trajectories. Meanwhile, a Reciprocal Control Barrier Function (RCBF) considering both relative position and relative velocity is designed. Both functions are incorporated into a quadratic programming solver to acquire optimal control inputs satisfying safety constraints in real time, and the feasibility of the optimization problem is strictly verified theoretically. Simulation results of 8-UAV formation demonstrate that the swarm can accomplish the formation construction and maintenance of cubic topology against internal and external disturbances. The minimum distance among followers remains at 0.75 m, which is higher than the safety threshold of 0.5 m. This value is remarkably superior to 0.37 m of the improved artificial potential field method and 0.10 m of the scheme without collision avoidance mechanism. The proposed strategy possesses comprehensive advantages in formation tracking accuracy, disturbance resistance performance and collision avoidance security, and exhibits favorable practical engineering application prospects.