Autonomous aerial swarms demonstrate significant potential for a range of applications, such as environmental monitoring, disaster response, and search-and-rescue operations. However, achieving safe and decentralized navigation in dynamic, cluttered environments remains a fundamental challenge, particularly under strict constraints of onboard sensing and computation. Classical modular pipelines suffer from latency accumulation and limited scalability, while fully end-to-end Reinforcement Learning (RL) approaches often face severe sim-to-real degradation and lack safety or stability guarantees. To address these challenges, this paper proposes a novel learning-based decentralized navigation framework that integrates a LiDAR-based RL policy with a Safety-assured Nonlinear Model Predictive Controller (SA-NMPC) for reliable execution. The proposed framework features a biologically-inspired decoupled hierarchical architecture: the RL front-end generates agile, short-horizon navigation commands based on raw Light Detection and Ranging (LiDAR) scans, while the SA-NMPC back-end ensures dynamically feasible tracking and active disturbance rejection. To ensure safe operation in dynamic scenes, an asynchronous dual-stream perception system is employed to enhance the capabilities of dynamic obstacle tracking and static map maintenance. The proposed framework has been validated through extensive simulation and real-world experiments, including the 2025 IEEE IROS Aerial Autonomy Challenge and multi-quadrotor swarm flights. The system demonstrates zero-shot sim-to-real transfer capability, robust performance in dynamic environments, and significant improvements over both classical and learning-based baselines.