AI Chat Paper
Note: Please note that the following content is generated by AMiner AI. SciOpen does not take any responsibility related to this content.
{{lang === 'zh_CN' ? '文章概述' : 'Summary'}}
{{lang === 'en_US' ? '中' : 'Eng'}}
Chat more with AI
PDF (6.3 MB)
Collect
Submit Manuscript AI Chat Paper
Show Outline
Outline
Show full outline
Hide outline
Outline
Show full outline
Hide outline
Open Access

Ensemble Knowledge Distillation for Federated Semi-Supervised Image Classification

School of Computer Science and Technology, Xidian University, Xi’an 710126, China
Engineering Research Center of Blockchain Technology Application and Evaluation, Ministry of Education, Xi’an 710126, China, and Key Laboratory of Smart Human-Computer Interaction and Wearable Technology of Shaanxi Province, Xi’an 710126, China
Show Author Information

Abstract

Federated learning is an emerging privacy-preserving distributed learning paradigm, in which many clients collaboratively train a shared global model under the orchestration of a remote server. Most current works on federated learning have focused on fully supervised learning settings, assuming that all the data are annotated with ground-truth labels. However, this work considers a more realistic and challenging setting, Federated Semi-Supervised Learning (FSSL), where clients have a large amount of unlabeled data and only the server hosts a small number of labeled samples. How to reasonably utilize the server-side labeled data and the client-side unlabeled data is the core challenge in this setting. In this paper, we propose a new FSSL algorithm for image classification based on consistency regularization and ensemble knowledge distillation, called EKDFSSL. Our algorithm uses the global model as the teacher in consistency regularization methods to enhance both the accuracy and stability of client-side unsupervised learning on unlabeled data. Besides, we introduce an additional ensemble knowledge distillation loss to mitigate model overfitting during server-side retraining on labeled data. Extensive experiments on several image classification datasets show that our EKDFSSL outperforms current baseline methods.

References

【1】
【1】
 
 
Tsinghua Science and Technology
Pages 112-123

{{item.num}}

Comments on this article

Go to comment

< Back to all reports

Review Status: {{reviewData.commendedNum}} Commended , {{reviewData.revisionRequiredNum}} Revision Required , {{reviewData.notCommendedNum}} Not Commended Under Peer Review

Review Comment

Close
Close
Cite this article:
Shang E, Liu H, Zhang J, et al. Ensemble Knowledge Distillation for Federated Semi-Supervised Image Classification. Tsinghua Science and Technology, 2025, 30(1): 112-123. https://doi.org/10.26599/TST.2023.9010156

2514

Views

169

Downloads

18

Crossref

16

Web of Science

16

Scopus

1

CSCD

Received: 27 October 2023
Revised: 10 December 2023
Accepted: 19 December 2023
Published: 11 September 2024
© The Author(s) 2025.

The articles published in this open access journal are distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/).