Being able to probe the polarization states in low-symmetry two-dimensional (2D) crystals is crucial for applications in polarization-encoded encryption and decryption. However, their polarization sensitivity is restricted by the inherent optical anisotropy. Here, we demonstrate a strategy where black phosphorus (BP) is stacked on Nb₂SiTe₄ to form a van der Waals heterostructure, realizing a self-powered, broadband, and highly polarization-sensitive photodetector. The atomically sharp and clean interface of BP/Nb₂SiTe₄ heterostructures forms a strong built-in electric field, which facilitates the separation of electron–hole pairs. Furthermore, angle-dependent Raman and absorption intensities reveal significant anisotropic optical response. The BP/Nb₂SiTe₄ photodetector exhibits a rapid response time and broadband detection capability from 360 to 2200 nm due to the efficient charge separation and strong interlayer coupling at the heterostructure interface. Notably, the BP/Nb₂SiTe₄ device also exhibits a large dichroic ratio (10.4) at zero voltage under 2200 nm light illumination. Furthermore, the proof-of-concept BP/Nb₂SiTe₄ photodetector possesses polarization-resolved imaging and polarization-encoded encryption/decryption capabilities, providing new insights for achieving low-power and multifunctional polarization-sensitive photodetectors.