Atomically thin two-dimensional (2D) van der Waals (vdW) layered semiconductors with dangling-bond-free surfaces and excellent electronic properties are considered as alternative channel materials to address the challenges faced by the miniature transistors at next-generation technology nodes. However, the scarcity of high-dielectric-constant (κ) vdW single-crystalline gate dielectrics that are well compatible with them poses an obstacle to the practical applications of 2D semiconductors. Here, we report the synthesis of high-κ vdW Bi₂TeO₅ single crystals via chemical vapor deposition, as well as their applications as gate dielectrics. The as-grown vdW Bi₂TeO₅ shows in-plane ferroelectricity, featuring a high Curie temperature and a remarkable dielectric constant (κ ≈ 48), which is currently the highest value reported for various vdW dielectrics. The 2D semiconductor-based transistors gated by Bi₂TeO₅ crystal achieve an on-off current ratio exceeding 10⁷, a near-Boltzmann-limit subthreshold swing, and a low gate leakage current. Furthermore, the three-dimensional-stacked complementary field-effect transistor inverters are constructed by integrating 2D vdW semiconductors and dielectrics, and exhibit a voltage gain of 4 and a power consumption of 193 pW at a low supply voltage of 0.3 V. Our work demonstrates that 2D vdW Bi₂TeO₅ is a promising high-κ single-crystalline gate dielectric and opens up a new opportunity for highly scalable, low-power 2D electronics.