Double-inertial bilevel optimization for hybrid deep learning: Accelerating chest X-ray classification

Convex bilevel optimization, in which an upper-level problem is optimized over the solution set of a lower-level convex program, naturally arises in machine learning tasks such as hyperparameter tuning, meta-learning, and regularized classification. Existing first-order bilevel solvers typically employ single-inertial acceleration, which can exhibit oscillatory behaviors and premature stagnation. In this paper, we propose the Double-Inertial proximal Gradient Method with Viscosity Approximation (DIPGM-VA), a novel algorithm that combines double-inertial extrapolation with viscosity approximation within a forward–backward splitting framework. We establish the strong convergence of the iterates to the unique solution of the bilevel problem in real Hilbert spaces under standard assumptions. As an application, we develop a hybrid deep learning architecture that integrates EfficientNetV2-B0 with an Extreme Learning Machine (ELM) classifier whose output weights are determined by solving a bilevel optimization problem via DIPGM-VA, and validate it on a pediatric chest X-ray pneumonia benchmark, addressing a leading cause of death among children under five in developing countries. Numerical experiments show that DIPGM-VA outperforms other state-of-the-art bilevel solvers by reaching a lower inner-level objective value within the same iteration budget. Concurrently, the hybrid model delivers a competitive F1-score of 0.9001 and accelerates the training process by an order of magnitude relative to standard end-to-end approaches.