Frequency Domain Features Based Improving Gradient Descent Optimization for Cyber-Physical-Social Intelligence

Cyber-Physical-Social Intelligence (CPSI) enhances the interpretability of Gradient Descent (GD) optimizers and improves image recognition models. To address the black-box nature of optimizers, we propose a new approach to map the GD optimizer to the frequency domain, allowing analysis of gradient variations at different frequencies. This approach aids in selecting optimal training and optimization strategies, offering a novel solution to the challenge of optimizer interpretability. Specifically, firstly, based on the transfer function of the gradient descent optimizer, the real-frequency characteristic and imaginary-frequency characteristic functions of the optimizer are derived for the first time, which provide a new perspective for the selection of the optimizer and the interpretability of the tuning parameter. Then, based on these characteristic functions, the learning properties of the optimizer in the frequency domain are analyzed for the first time, which provides an important reference for improving the performance of the optimizer in real application problems. Finally, the effectiveness of the frequency domain modulation properties is verified through convex and non-convex optimization problems. Experimental results show that the proposed theory not only improves the recognition accuracy, convergence speed, and stability, but also extends its scope in practical applications.