Atomically thin transition metal dichalcogenides (TMDs) have distinct opto-electronic properties including enhanced luminescence and high on-off current ratios, which can be further modulated by making more complex TMD heterostructures. However, resolution limits of conventional optical methods do not allow for direct nanoscale optical-structural correlation measurements in these materials, particularly of buried interfaces in TMD heterostructures. Here we use, for the first time, electron beam induced cathodoluminescence in a scanning transmission electron microscope (CL-STEM) to measure optical properties of monolayer TMDs (WS₂, MoS₂ and WSSe alloy) encapsulated between layers of hBN. We observe dark areas resulting from localized (~ 100 nm) imperfect interfaces and monolayer folding, which shows that the intimate contact between layers in this application-relevant heterostructure is required for proper inter layer coupling. We also realize a suitable imaging method that minimizes electron-beam induced changes and provides measurement of intrinsic properties. To overcome the limitation of small electron interaction volume in TMD monolayer (and hence low photon yield), we find that encapsulation of TMD monolayers with hBN and subsequent annealing is important. CL-STEM offers to be a powerful method to directly measure structure-optical correspondence in lateral or vertical heterostructures and alloys.