Unifying Data Effectiveness Assessment in User Churn Detection: An Indicator-Assisted Framework

As Artificial Intelligence (AI) demonstrates an impressive performance across various tasks, its ability to process diverse data types has been widely adopted. However, AI models often lack transparency and interpretability, making it challenging to identify the most effective data categories and models for specific contexts. Current strategies typically rely on common sense or exhaustive searches, both of which are inefficient and costly. To fill this gap, we propose the INdicator-assisted FramewOrk (INFO). INFO enables researchers and practitioners to directly identify effective data categories and guide model selection for tasks involving multiple data categories. By assessing the inherent signal strength of the data, INFO provides a proxy for evaluating task performance, thereby reducing the need for time-consuming model trials and reproducibility efforts. We validate the effectiveness of INFO using real-world datasets across five different domains, focusing on the critical task of user churn detection as the case study. Our experiments show that INFO can reveal the efficacy of data categories before large-scale trials. INFO shows promise as an automated data engineering methodology that clarifies the relationship between model performance and dataset characteristics.