In the field of mining engineering, the drill-and-blast method is a prevalent excavation technique. The structural planes of the rock mass directly influence the propagation and attenuation of blasting stress waves, thereby rendering the precise and efficient determination of rock layer interface positions essential for the operational efficiency and safety of drill-and-blast operations. This study leverages a self-developed indoor digital drilling test platform to conduct cutting and drilling experiments on layered rock-like specimens, obtaining drilling operational parameters and vibration response signals. A prediction model for identifying layered rock mass interfaces is established using the GA-K-means algorithm, with the CH index employed as a fitness metric to enhance the model′s global search capability. A comparative analysis demonstrates that the cumulative errors for drilling vibration response and drilling operational parameters are 2.02 mm and 6.08 mm, respectively, indicating that the former exhibits greater sensitivity to variations in rock layers and provides more accurate interface identification outcomes. Drilling parameters are prone to significant deviations due to uneven rock breaking by the drill bit, while changes in these parameters notably influence vibration response signals in rock layer strength. Predicted interface positions often occur ahead of actual interfaces, revealing their potential for early prediction. This paper proposes a method based on feature weighting and cluster analysis to improve the accuracy and efficiency of rock layer interface identification. Compared to the K-means algorithm, the GA-K-means algorithm achieves over 31% higher identification accuracy. The results demonstrate that the GA-K-means rock layer interface identification model can effectively extract key rock layer information from complex drilling parameters, providing valuable references for geological exploration.