We investigate the space-time decay rates of solutions to the 3D Cauchy problem of the compressible Oldroyd-B system with diffusive properties and without viscous dissipation. The main novelties of this paper involve two aspects: On the one hand, we prove that the weighted rate of $k$-th order spatial derivative (where $0\le k\le 3$) of the global solution $(\rho ,u,\eta ,\tau )$ is $t^{-\frac{3}{4}+\frac{k}{2}+\gamma }$ in the weighted Lebesgue space $L_{\gamma }^2$. On the other hand, we show that the space-time decay rate of the $m$-th order spatial derivative (where $m\in \left[0,2\right]$) of the extra stress tensor of the field in $L_{\gamma }^2$ is $(1+t{)}^{-\frac{5}{4}-\frac{m}{2}+\gamma }$, which is faster than that of the velocity. The proofs are based on delicate weighted energy methods and interpolation tricks.