With the deep integration of edge computing, 5G and Artificial Intelligence of Things (AIoT) technologies, the large-scale deployment of intelligent terminal devices has given rise to data silos and privacy security challenges in sensing-computing fusion scenarios. Traditional federated learning (FL) algorithms face significant limitations in practical applications due to client drift, model bias, and resource constraints under non-independent and identically distributed (Non-IID) data, as well as the computational overhead and utility loss caused by privacy-preserving techniques. To address these issues, this paper proposes an Efficient and Privacy-enhancing Clustering Federated Learning method (FedEPC). This method introduces a dual-round client selection mechanism to optimize training. First, the Sparsity-based Privacy-preserving Representation Extraction Module (SPRE) and Adaptive Isomorphic Devices Clustering Module (AIDC) cluster clients based on privacy-sensitive features. Second, the Context-aware In-cluster Client Selection Module (CICS) dynamically selects representative devices for training, ensuring heterogeneous data distributions are fully represented. By conducting federated training within clusters and aggregating personalized models, FedEPC effectively mitigates weight divergence caused by data heterogeneity, reduces the impact of client drift and straggler issues. Experimental results demonstrate that FedEPC significantly improves test accuracy in highly Non-IID data scenarios compared to FedAvg and existing clustering FL methods. By ensuring privacy security, FedEPC provides an efficient and robust solution for FL in resource-constrained devices within sensing-computing fusion scenarios, offering both theoretical value and engineering practicality.