In this paper, a high order compact finite difference is established for the time multi-term fractional sub-diffusion equation. The derived numerical differential formula can achieve second order accuracy in time and four order accuracy in space. A unconditionally stable and convergent difference scheme is presented, and a rigorous proof for the stability and convergence is given. Numerical results demonstrate the efficiency of the proposed difference schemes.