The burgeoning demand for unmanned aerial vehicles (UAVs) across diverse domains can be attributed to their high flexibility, ease of deployment and low operational costs. Concomitantly, rapid advancements in reinforcement learning have emerged as a viable avenue for augmenting the autonomy of UAVs. This paper provides a comprehensive overview of the foundational concepts and methodologies of reinforcement learning and taxonomizes its applications in UAV decision-making into three primary categories: fundamental tasks encompassing obstacle avoidance and path planning, advanced tasks involving cooperative control, and complex tasks requiring adversarial decision-making. Additionally, the challenges associated with implementing reinforcement learning in UAV applications are critically examined. In the final section, we envision future research directions and provide a comprehensive summary of the study. This will assist practitioners and researchers in selecting appropriate reinforcement learning algorithms for their drone mission applications.