Despite significant advancements in 3D building reconstruction from point clouds, existing methods face persistent challenges in generating compact building structural representations, particularly under occluded, noisy, or incomplete data conditions. Current approaches suffer from three critical limitations: (1) heavy reliance on high-quality point clouds with minimal noise and occlusion, (2) geometric inconsistencies and topological errors during primitive assembly, and (3) inability to robustly convert geometric primitives into watertight models. To overcome these limitations, we propose a novel framework that combines structure line-aware Markov random field (MRF) refinement with watertight-oriented primitive assembly, achieving robust reconstruction of building primary structures. Our method introduces three key innovations: First, a RANSAC-assisted orthogonal projection method combined with a lightweight L-CNN network is employed to extract line primitives from 2D façade images, reducing the dependency on scarce 3D annotations. Second, a structure line-aware MRF model that integrates geometric constraints and adjacency consistency is constructed, resolving fragmented or noisy lines. Finally, geometrical consistent boundary polygons are generated based on structure lines, and a dual regularization pipeline (i.e. the buffer box and slicing methods) is introduced to assemble them, enabling watertight assembly even under severe occlusions. Extensive evaluations on diverse datasets (TLS, wearable laser scanning, and photogrammetric point clouds) demonstrate that our method achieves a recall rate of over 90% and a regularization rate exceeding 95% for structure lines, indicating the high efficiency of extraction and refinement of the structure lines. Moreover, the reconstructed models exhibit less than 0.07 m average distance and 0.09 m RMSE, outperforming state-of-the-art methods (e.g. CA, Polyfit, KSR) in geometric fidelity. This validates the effectiveness in recovering building's primary structure, offering an effective solution for building reconstruction in smart city and digital twin applications.