Empowering Large Language Models for Relational Data Cleaning and Integration

Relational data, as one of the most common data forms, are widely applied in fields such as recommendation systems, medical diagnosis, and traffic prediction. Relational data formats often face issues pertaining to incompleteness and deficiencies in quality, necessitating rigorous cleansing and integration processes to unleash their potential. However, different downstream tasks and data characteristics impose various requirements on the cleansing and integration processes. Currently, there is an absence of unified and effective methods. Large Language Models (LLMs), with their extensive real-world knowledge and strong natural language processing abilities, are emerging as robust tools for cleansing and integrating relational data. This study investigates techniques for tackling relational data cleaning and integration challenges, and proposes a comprehensive framework based on LLMs. The framework uses LLMs for problem-solving, integrating multi-granularity table representation models and alignment networks to improve task-related table understanding. Moreover, the framework facilitates straightforward deployment across various LLMs. To assess the efficacy of the LLMs and the proposed framework, this study performs experiments on the task of imputing missing values utilizing four real-world datasets. The experiments evaluate the performance of a proprietary LLM, an open-source LLM that has been fine-tuned, and the framework itself, comparing and analyzing their results with existing techniques. The experiments demonstrate that LLMs significantly outperform traditional methods in these tasks. Furthermore, ablation experiments are conducted to verify the effectiveness of the framework.