Laser scanning technology plays a pivotal role in diverse applications. However, challenges arise when fitting geometric point clouds or detecting objects using conventional RANSAC due to the dense concentration or uneven distribution of noise points along the boundaries of geometric objects. In this study, we propose a novel framework for random sample consensus (RANSAC) fitting, incorporating a bi-objective scoring function (BSFM). Initially, we introduce an innovative classification technique to optimize the accurate distinction between inliers and outliers within the geometry edge region. Subsequently, we construct a new bi-objective scoring function and iteratively optimize the model parameters of RANSAC, including error thresholds. To validate our method, comprehensive numerical tests are conducted under varying levels of noise pollution. The results demonstrate the superior performance of our approach compared to the three classical methods in terms of geometric fitting accuracy and robustness. Furthermore, when applied to the inspection of real reinforcing steel mesh, the evaluation indexes of our BSFM-RANSAC method surpass those of the conventional RANSAC technique.