Federated Learning (FL) is an emerging distributed machine learning paradigm that provides privacy guarantees for training robust models on distributed clients. The primary challenge of FL is data heterogeneity, which slows down model convergence and degrades model performance. Knowledge distillation has recently demonstrated effectiveness in addressing this challenge. However, these approaches neglect the statistical heterogeneity in local models and the uncertainty of the data distribution in the global model, which results in the ensemble knowledge cannot be fully utilized to guide local model learning. In this work, we propose an unsupervised knowledge distillation method migrating the local class-level pseudo-data sample scheme in the server for fine-tuning the global model. Specifically, we provide the conditional autoencoder for each client to maintain a dynamic generator in the server, which ensembles the client’s class-level information. The proposal produces an auxiliary dataset representing the global class-level distribution to regulate the local model as an inductive knowledge bias, and employs unsupervised knowledge distillation to enhance the aggregated model’s performance. The extensive experiments show that our proposal significantly outperforms the current state-of-the-art FL algorithms and can be integrated as a flexible plugin into existing FL optimization algorithms to enhance model performance.

The emergence of coordinated and consistent macro behavior among self-interested individuals competing for limited resources represents a central inquiry in comprehending market mechanisms and collective behavior. Traditional economics tackles this challenge through a mathematical and theoretical lens, assuming individuals are entirely rational and markets tend to stabilize through the price mechanism. Our paper addresses this issue from an econophysics standpoint, employing reinforcement learning to construct a multi-agent system modeled on minority games. Our study has undertaken a comparative analysis from both collective and individual perspectives, affirming the pivotal roles of reward feedback and individual memory in addressing the aforementioned challenge. Reward feedback serves as the guiding force for the evolution of collective behavior, propelling it towards an overall increase in rewards. Individuals, drawing insights from their own rewards through accumulated learning, gain information about the collective state and adjust their behavior accordingly. Furthermore, we apply information theory to present a formalized equation for the evolution of collective behavior. Our research supplements existing conclusions regarding the mechanisms of a free market and, at a micro level, unveils the dynamic evolution of individual behavior in synchronization with the collective.