Land resources can greatly contribute to the national food security in sustainable agriculture. The rational use of land is often required for the better match of land functions. However, the garden land can be found under the mountain and farmland on the hill. It is of great significance to promote the replacement of farmland with the forests and gardens, according to the reginal land and climate conditions. This study aims to optimize the spatial distribution of agricultural land, particularly for the promotion of arable land preservation and sustainable use of farmland. The replacement potential of farmland with forests was calculated after optimization. A case study was taken as the Nanfeng County, Jiangxi Province, China. An evaluation index system was then constructed for the arable land suitability from four aspects: land quality, geographical location, ecological conditions and spatial pattern. The four-quadrant method was also used to measure the potential with the slope and arable land suitability as the dominant factors. The results show that: 1) The suitability of farmland was generally excellent, but there was the low suitability level of local farmland. The full use of land was difficult to realize in the study area during this time; The relatively small areas of arable suitability in Class IV and V forests were accounted for 65.39% of the total; There were the relatively large areas of arable suitability in the Class I, Ⅱ and Ⅲ fields, which were accounted for 84.73% of the total; Most lands were favorable in agricultural production. But some of the fields shared the less arable land to develop. 2) There was the uneven distribution in the replacement potential with farmland and forest. Specifically, the potential replacement was obtained with the farmland total of 15 437.86 hm², the forest land of 6 548.17 hm² under the mountain, and 8 889.69 hm² was the field under the hill; The total potential of forest land to be replaced was 778.91 hm², of which 69.70 hm² was farmland on the mountain, and 709.21 hm² was the garden land on the mountain to be adjusted. 3) The replacement potential shared the outstanding spatial differentiation. Among them, the potential suitability for the replacement with farmland was distributed mainly in Sangtian, Baishe and Laixi Town, which were concentrated in the central valley plain; The potential suitability for the replacement with the forest was distributed in the hilly and mountainou areas, such as Fufang, Sanxi, Taihe and Zixiao Town. There were more areas in the east and west and less in the center as a whole. 4) It was recommended to formulate the differentiated promotion strategies, according to the different scenarios after land-type adjustment in the regional conditions. This identification was also different from the previous measurement of replacement potential using suitability evaluation. A potential identification was constructed for the farmland, forest and field replacement that combined slope and farm capacity. The mismatch space of farmland, forest and field was identified over the slope, and then screened the replacement plots. Farm evaluation was used to effectively clarify the direction, potential and optimizing path for the layout of agricultural land. A broader scale was utilized to identify the replacement potential of farmland, forests and fields. Field surveys were also carried out to further verify the identification. Additionally, various challenges were still remained in the conversion of farmland into the forests and gardens, including the inadequate policy support, conflicts over land ownership, unequal distribution of benefits, and low willingness among farmers. Theoretical assessments of potential can also contribute to the optimal adjustment in the layout of agricultural land.