Novel inertial stochastic Bregman inexact ADMMs for solving large-scale nonconvex and nonsmooth optimization without relying on the Kurdyka–Łojasiewicz property

As the scale of optimization problems expands, the performance of the alternating direction method of multipliers (ADMM) exhibits a significant downward trend. In this paper, aiming at solving nonconvex, nonsmooth optimization problems under large-scale linear constraints, we proposed a unified framework of novel stochastic inexact ADMMs incorporating inertial terms and Bregman distances. By fusing the Bregman distance with inertial acceleration techniques, the framework not only covers stochastic gradient descent and existing variance-reduced gradient estimation techniques such as the stochastic variance-reduced gradient and stochastic recursive gradient, but also allows for a more flexible double-step strategy in convergence analysis. Without depending on the Kurdyka–Łojasiewicz property and under some suitable mild conditions, we demonstrated global convergence of this unified framework, and showed that it achieves a sublinear convergence rate of $\mathcal{O}(1/\mathbb{K})$, where $\mathbb{K}$ is the number of iterations. Further, under error bound conditions, the linear convergence rate of the stochastic inexact ADMMs was established. Finally, the effectiveness of stochastic inexact ADMMs for solving some nonsmooth and nonconvex problems was verified by numerical experiments on the graphically guided fusion LASSO problems.