Decent directions generator conjugate gradient method with its application to train a two-layer neural network model

In the last decades, conjugate gradient methods have gained important applications in various scientific areas due to their low memory requirements and ability to solve problems of high dimensions. When analyzing a conjugate gradient method, the descent property of the search directions is always required, as it ensures that the search for the minimizer is in the correct direction. In this paper, we proposed a conjugate gradient method that always generates descent search directions under all line searches techniques. Moreover, we established the global convergence of the proposed method when it is applied under Wolfe or strong Wolfe line search. At the same time, to show the performance of the proposed method in practical computation, we compared it with other well-known methods and then applied it to train two-layer neural network models. The numerical results show that the proposed method is efficient.