Rare-earth (RE) based luminescent probes exhibit rich optical properties including upconversion and down-conversion luminescence spanning a broad spectral range from 300 to 3,000 nm, and have generated great scientific and practical interest from telecommunication to biological imaging. While upconversion nanoparticles have been investigated for decades, down-conversion luminescence of RE-based probes in the second near-infrared (NIR-II, 1,000-1,700 nm) window for in vivo biological imaging with sub-centimeter tissue penetration and micrometer image resolution has come into light only recently. In this review, we present recent progress on RE-based NIR-II probes for in vivo vasculature and molecular imaging with a focus on Er3+-based nanoparticles due to the down-conversion luminescence at the long-wavelength end of the NIR-II window (NIR-IIb, 1,500-1,700 nm). Imaging in NIR-IIb is superior to imaging with organic probes such as ICG and IRDye800 in the ~ 800 nm NIR range and the 1,000-1,300 nm short end of NIR-II range, owing to minimized light scattering and autofluorescence background. Doping by cerium and other ions and phase engineering of Er3+-based nanoparticles, combined with surface hydrophilic coating optimization can afford ultrabright, biocompatible NIR-IIb probe towards clinical translation for human use. The Nd3+-based probes with NIR-II emission at 1,050 and 1,330 nm are also discussed, including Nd3+ doped nanocrystals and Nd3+-organic ligand complexes. This review also points out future directions for further development of multi-functional RE NIR-II probes for biological imaging.