Federated Medical Image Segmentation with Attention-Based Structure Knowledge Ensemble

Federated medical image segmentation holds potential for training a global model to facilitate collaborative multi-site medical image analysis. However, the challenge of non-independent and non-identically distributed multi-site medical data complicates the task for Federated Learning (FL) methods. Heterogeneous feature distributions, resulting from varying imaging acquisition protocols or scanner vendors, often hinder the global model’s performance. While some previous studies have tackled the non-iid issue, they primarily focus on image classification tasks and cannot be readily applied to segmentation. In this paper, we present a novel method, Federated Attention-based Map Ensemble (FedAME), tailored to overcome the unique challenges posed by federated image segmentation. Firstly, FedAME introduces a novel knowledge type called attention-based structural knowledge, effectively harnessing the underlying semantic region while filtering out noisy regions. Secondly, to align local models before aggregation, we incorporate structure knowledge ensembles into local training, considering the drift of semantic information between global and local models. Thirdly, we propose a dynamic policy for adaptively generating consensus structural knowledge. It enhances collaboration among sites sharing similar feature distributions and effectively mitigates the negative contributions of dissimilar sites. Comprehensive evaluations and ablation studies conducted across multiple datasets show that FedAME outperforms state-of-the-art methods.