Unmanned Aerial Vehicle (UAV) assisted federated learning enables on-edge model training, but its effectiveness depends on sustainable client participation through well-designed incentive mechanisms. Existing approaches based on economic models provide theoretical guarantees under restrictive assumptions, while Reinforcement Learning (RL) methods adapt to dynamics but lack provable incentive compatibility. We propose an adaptive privacy-aware incentive mechanism that integrates contract theory with Multi-Agent RL (MARL). The contract design provides a truthful initialization under privacy heterogeneity, while MARL adaptively refines incentives in dynamic environments. An Incentive Compatibility (IC) regularized optimization further ensures policy convergence and robustness. Experiments in UAV-assisted FL show that our method improves social welfare by up to 35% and participant engagement by 45% over state-of-the-art baselines, while maintaining strong privacy guarantees.