A learning-guided optimization approach, i.e., a $Q$-Learning-guided Memetic Algorithm (QLMA), is proposed to solve path planning problems in unknown environments. Traditional path planning algorithms in unknown environments have the disadvantages of low efficiency and stability problems. Integrating learning strategies into a memetic algorithm is a natural way to enhance search performance. The proposed algorithm can be divided into two stages: learning and optimization. A $Q$-learning algorithm implements the learning stage, while the optimization stage is implemented by hybridization of a Genetic Algorithm (GA) and Simulated Annealing (SA) algorithm, i.e., a memetic algorithm. The learning stage aims to acquire information from current search states. The $Q$-learning algorithm’s environmental learning capability is formed by rewards received from changes in distance to the target. Therefore, the path strategy could be adjusted continually, and effective paths could be identified. The optimization stage aims to utilize the obtained information and enhance the quality of known paths. A GA improves the global search ability, while an SA algorithm enhances the local search ability. The experimental study was conducted on 30 test scenarios with three types of index points. The proposed algorithm outperforms the other two algorithms significantly in the test scenarios.