Large language models (LLMs) have emerged as the cornerstone of various natural language processing activities, enabling everything from chatbots to text classification and summarization. However, using LLMs presents some significant challenges, most notably the threat of intellectual property infringement from the exposure of the entire model and the excessive communication and storage overhead associated with their large size. We propose KDLLM, a novel knowledge distillation-based framework for efficient and compact LLMs to address these challenges. KDLLM transfers the performance of a large teacher LLM to a significantly smaller student model with high performance similarity to its teacher, while obscuring architectural and parameter-level details to protect the intellectual property of the original model. The resulting student model substantially reduces the memory footprint and transmission overhead, making it amenable to deployment in bandwidth-constrained or security-sensitive environments. Comprehensive experiments demonstrate that KDLLM achieves robust performance preservation and boosts copyright protection and communication efficiency.

Deep learning based techniques are broadly used in various applications, which exhibit superior performance compared to traditional methods. One of the mainstream topics in computer vision is the image super-resolution task. In recent deep learning neural networks, the number of parameters in each convolution layer has increased along with more layers and feature maps, resulting in better image super-resolution performance. In today’s era, numerous service providers offer super-resolution services to users, providing them with remarkable convenience. However, the availability of open-source super-resolution services exposes service providers to the risk of copyright infringement, as the complete model could be vulnerable to leakage. Therefore, safeguarding the copyright of the complete model is a non-trivial concern. To tackle this issue, this paper presents a lightweight model as a substitute for the original complete model in image super-resolution. This research has identified smaller networks that can deliver impressive performance, while protecting the original model’s copyright. Finally, comprehensive experiments are conducted on multiple datasets to demonstrate the superiority of the proposed approach in generating super-resolution images even using lightweight neural networks.