At present, the actual mechanism of the photoluminescence (PL) of fluorescent carbon dots (CDs) is still an open debate among researchers. Because of the variety of CDs, it is highly important to summarize the PL mechanism for these kinds of carbon materials; doing so can guide the development of effective synthesis routes and novel applications. This review will focus on the PL mechanism of CDs. Three types of fluorescent CDs were involved: graphene quantum dots (GQDs), carbon nanodots (CNDs), and polymer dots (PDs). Four reasonable PL mechanisms have been confirmed: the quantum confinement effect or conjugated π-domains, which are determined by the carbon core; the surface state, which is determined by hybridization of the carbon backbone and the connected chemical groups; the molecule state, which is determined solely by the fluorescent molecules connected on the surface or interior of the CDs; and the crosslink-enhanced emission (CEE) effect. To give a thorough summary, the category and synthesis routes, as well as the chemical/physical properties for the CDs, are briefly introduced in advance.