In this paper, we consider a complete strongly stable hypersurface $M^n$ with constant mean curvature (CMC) $H$ in $N^{n+1}(\kappa )$, where $2\le n\le 5$, and prove that $M^n$ is totally umbilical if $H^2+\kappa >0$ and the $L^2$ norm curvature satisfies some growth conditions. In addition, we obtain that a complete stable hypersurface $M^n$ in $N^{n+1}(\kappa )(n=3,4,5)$ with CMC is a geodesic sphere if $-H^2<\kappa \le 0$ and $M^n$ has some finite $L^p$ norm curvature.