In this paper, the definition of the hyper-dual Moore-Penrose generalized inverse of a hyper-dual matrix is introduced. Characterizations for the existence of the hyper-dual Moore-Penrose generalized inverse are given, and a formula for the hyper-dual Moore-Penrose generalized inverse is presented whenever it exists. Least-squares properties of the hyper-dual Moore-Penrose generalized inverse are discussed by introducing a total order of hyper-dual numbers. We also introduce the definition of a dual matrix of order $n$. A necessary and sufficient condition for the existence of the Moore-Penrose generalized inverse of a dual matrix of order $n$ is given.