A novel hybrid model combining a convolutional neural network (CNN) and a low-complexity Transformer network is introduced for predicting lung cancer response to neoadjuvant chemoimmunotherapy using computed tomography scans. This approach is crucial as it assists clinicians in identifying patients likely to benefit from treatment and in assessing their prognosis. The model employs channel splitting to minimize parameter count. It then leverages both CNN for local feature extraction and a streamlined Transformer for global feature comprehension. To enhance efficiency, a novel self-attention mechanism is implemented, focusing on feature aggregation and element-wise multiplication. To address the different semantic meanings of features, an attention-based module is designed to seamlessly integrate features from both networks, employing a process of coarse fusion, attention computation, and fine fusion. When evaluated with data from 232 lung cancer patients who have undergone neoadjuvant chemoimmunotherapy, the model demonstrates exceptional performance, achieving a Dice score of 47.04% and a 95.00% Hausdorff distance of 25.12 mm, outperforming existing methods. Additionally, it has only 2.91×10⁶ parameters and 52.95×10⁹ floating point operations. Moreover, the model’s predictive accuracy in tumor diameter estimation is beneficial for treatment planning. Its robustness is further validated through its application in stroke lesion prediction, indicating its broad applicability.