Thermoelectric (TE) materials are receiving increasing attention due to their ability to directly converting heat to electricity. They are used to harvest electrical energy from the wasted heat in order to increase the efficiency of global energy. Polymer-based TE materials are particularly fascinating to wearable and mobile devices due to their low density, good flexibility, and low toxicity. This review summarizes the recent breakthroughs and optimization strategies of polymer-based TE materials. Among a large number of different organic TE materials, those with remarkable TE performance are selected and divided into three categories, which are poly(3, 4-ethylenedioxythiophene) derivatives, carbon nanotube/conductive polymer composites, and inorganic semiconductive nanomaterial/polymer composites. The effect of components and structures on the power factor are presented and discussed. Finally, some challenges are described and suggestions are provided for preparing the next-generation of polymer-based materials with high TE performance.