This paper mainly discusses the non-zero-sum Nash differential games for stochastic differential equations (SDEs) involving time-varying coefficient and infinite Markov jumps. First of all, a necessary and sufficient conditions for the existence of Nash equilibrium strategies is given, which turns the non-zero-sum Nash differential games into solving the equations that are composed of countable coupled generalized differential Riccati equations (CGDREs). As an application, a unified treatment is presented for $H_2$, $H_{\mathrm{\infty }}$, and $H_2/H_{\mathrm{\infty }}$ control by the Nash game approach, which can reveal the relationship among these three problems. Furthermore, the theoretical results are used to solve a numerical example.