PbS colloidal quantum dots (QDs) show great promise for short-wave infrared (SWIR) photodetection due to their tunable photoresponse and cost-effective solution processability, positioning them as a strong competitor to InGaAs technologies. Inverted device architectures, essential for compatibility with complementary metal-oxide-semiconductor (CMOS) readout circuits, face performance challenges due to limitations in the hole transport layer (HTL), such as porous NiO_x structures that cause surface recombination at low annealing temperatures. To overcome these challenges, herein, we develop a multi-HTL strategy integrating NiO_x, 1,2-ethanedithiol (EDT)-treated PbS, and self-assembled monolayers (SAMs) including [2-(9H-carbazol-9-yl)ethyl]phosphonic acid (2PACz) and [2-(3,6-dimethoxy-9H-carbazol-9-yl)ethyl]phosphonic acid (MeO-2PACz), significantly boosting the performance of inverted PbS QD photodetectors, with full-fullerene-based electron transport layers (ETLs). We confirm that the SAMs can effectively block electron transfer and passivate surface defects between the HTL and active layer, with 2PACz achieving an external quantum efficiency of 53% at 1200 nm and MeO-2PACz reducing dark current to 220 nA/cm², yielding a specific detectivity of 1.64 × 10¹² Jones, which represents the highest reported value under similar testing conditions and in this spectral region. This multi-HTL strategy enables high-performance SWIR imaging compatible with CMOS and thin-film transistor (TFT) circuits, advancing QD-based photodetection technologies.