Due to the attractive potential in avoiding the elaborate definition of anchor attributes, anchor-free-based deep learning approaches are promising for object detection in remote sensing imagery. CornerNet is one of the most representative methods in anchor-free-based deep learning approaches. However, it can be observed distinctly from the visual inspection that the CornerNet is limited in grouping keypoints, which significantly impacts the detection performance. To address the above problem, a novel and effective approach, called GroupNet, is presented in this paper, which adaptively groups corner specific to the objects based on corner embedding vector and corner grouping network. Compared with the CornerNet, the proposed approach is more effective in learning the semantic relationship between corners and improving remarkably the detection performance. On NWPU dataset, experiments demonstrate that our GroupNet not only outperforms the CornerNet with an AP of 12.8%, but also achieves comparable performance to considerable approaches with 83.4% AP.