In this paper, we classified Hadamard $2$- $(51,25,12)$ designs having a non-abelian automorphism group of order 10, i.e., ${\mathrm{F}\mathrm{r}\mathrm{o}\mathrm{b}}_{10}\cong Z_5:Z_2$, where a subgroup isomorphic to $Z_2$ fixed setwise all the $Z_5$-orbits of the sets of points and blocks. Furthermore, we classified $2$- $(59,29,14)$ designs having a non-abelian automorphism group of order 14, i.e., ${\mathrm{F}\mathrm{r}\mathrm{o}\mathrm{b}}_{14}\cong Z_7:Z_2$, where a subgroup isomorphic to $Z_2$ fixed setwise all the $Z_7$-orbits of the sets of points and blocks. We also showed that there was no Hadamard $2$- $(59,29,14)$ design with automorphism group ${\mathrm{F}\mathrm{r}\mathrm{o}\mathrm{b}}_{21}\cong Z_7:Z_3$, where a subgroup of order 3 fixed setwise all the $Z_7$-orbits of points and blocks. Additionally, we used Hadamard $2$- $(59,29,14)$ designs obtained to construct ternary linear codes and linear codes over the finite field of order 5. We constructed ternary self-dual codes and self-dual codes over the finite field of order 5 from the corresponding Hadamard matrices of order 60.