Due to the uneven density of the Earth’s atmosphere, the light propagation path bends, destroying the collinearity condition of the ground object, camera projection center, and image point and introducing the atmospheric refraction error of optical satellite. The atmospheric refraction error seriously affects the geometric positioning accuracy and restricts the application of remote sensing imagery. This study proposes a novel and generalized atmospheric refraction correction method based on the rational function model (RFM) to compensate for refraction errors in various optical satellites without the complex satellite ephemeris and protected camera parameters. By using globally measured atmospheric parameters and the imaging characteristics of optical satellites, global atmospheric refraction indices with 400 height layers were stored. A compensation model of atmospheric refraction error was proposed, based on a limited number of key points, to improve processing efficiency. Based on the projection relationship in optical satellite imaging, the satellite position and object direction of key points were determined through backward calculation of the RFM, eliminating the need for complex and undisclosed satellite auxiliary data. Atmospheric refraction error was compensated by conducting iterative geometric positioning and refraction correction using an atmospheric model with 400 height layers. Experimental results show that the proposed method can be applied to sub-meter-level resolution and large swath-wide optical images for atmospheric refraction error correction. The average processing time is less than 2 s. Moreover, the improvement in geometric positioning accuracy of optical images ranges from 0.057 m to 2.985 m. This method is as accurate as the refraction correction method using a rigorous model.