Rice panicle architecture affects grain number per panicle and thereby grain yield. Many genes involved in control of panicle architecture have been identified in the past decades. According to their effect on phenotype, these genes are divided into three categories: panicle branch and lateral spikelets, multifloret spikelets, and panicle type. We review these genes, describe their genetic regulatory network, and propose a strategy for using them in rice breeding. These findings on rice panicle architecture may facilitate related studies in other crops.