A Review on Penetration Testing for Privacy of Deep Learning Models

As deep learning (DL) models are increasingly deployed in sensitive domains (e.g., healthcare), concerns over privacy and security have intensified. Conventional penetration testing frameworks, such as OWASP and NIST, are effective for traditional networks and applications but lack the capabilities to address DL-specific threats, such as model inversion, membership inference, and adversarial attacks. This review provides a comprehensive analysis of penetration testing for the privacy of DL models, examining the shortfalls of existing frameworks, tools, and testing methodologies. Through systematic evaluation of existing literature and empirical analysis, we identify three major contributions: (i) a critical assessment of traditional penetration testing frameworks’ inadequacies when applied to DL-specific privacy vulnerabilities, (ii) a comprehensive evaluation of state-of-the-art privacy-preserving methods and their integration with penetration testing workflows, and (iii) the development of a structured framework that combines reconnaissance, threat modeling, exploitation, and post-exploitation phases specifically tailored for DL privacy assessment. Moreover, this review evaluates popular solutions such as IBM Adversarial Robustness Toolbox and TensorFlow Privacy, alongside privacy-preserving techniques (e.g., Differential Privacy, Homomorphic Encryption, and Federated Learning), which we systematically analyze through comparative studies of their effectiveness, computational overhead, and practical deployment constraints. While these techniques offer promising safeguards, their adoption is hindered by accuracy loss, performance overheads, and the rapid evolution of attack strategies. Our findings reveal that no single existing solution provides comprehensive protection, which leads us to propose a hybrid approach that strategically combines multiple privacy-preserving mechanisms. The findings of this survey underscore an urgent need for automated, regulation-compliant penetration testing frameworks specifically tailored to DL systems. We argue for hybrid privacy solutions that combine multiple protective mechanisms to ensure both model accuracy and privacy. Building on our analysis, we present actionable recommendations for developing adaptive penetration testing strategies that incorporate automated vulnerability assessment, continuous monitoring, and regulatory compliance verification.