In this paper, we are concerned with optimal decay rates of higher–order derivatives of the smooth solutions to the $3D$ compressible nematic liquid crystal flows. The main novelty of this paper is three–fold: First, under the assumptions that the initial perturbation is small in $H^N$–norm $(N\ge 3)$ and bounded in $L^1$–norm, we show that the highest–order spatial derivatives of density and velocity converge to zero at the $L^2$–rates is $(1+t{)}^{-\frac{3}{4}-\frac{N}{2}}$, which are the same as ones of the heat equation, and particularly faster than the $L^2$–rate $(1+t{)}^{-\frac{1}{4}-\frac{N}{2}}$ in [J.C. Gao, et al., J. Differential Equations, 261: 2334-2383, 2016]. Second, if the initial data satisfies some additional low frequency assumption, we also establish the lower optimal decay rates of solution as well as its all–order spatial derivatives. Therefore, our decay rates are optimal in this sense. Third, we prove that the lower bound of the time derivatives of density, velocity and macroscopic average converge to zero at the $L^2$–rate is $(1+t{)}^{-\frac{5}{4}}$. Our method is based on low-frequency and high-frequency decomposition and energy methods.