Interpretable Detection of Malicious Behavior in Windows Portable Executables Using Multi-Head 2D Transformers

Windows malware is becoming an increasingly pressing problem as the amount of malware continues to grow and more sensitive information is stored on systems. One of the major challenges in tackling this problem is the complexity of malware analysis, which requires expertise from human analysts. Recent developments in machine learning have led to the creation of deep models for malware detection. However, these models often lack transparency, making it difficult to understand the reasoning behind the model’s decisions, otherwise known as the black-box problem. To address these limitations, this paper presents a novel model for malware detection, utilizing vision transformers to analyze the Operation Code (OpCode) sequences of more than 350000 Windows portable executable malware samples from real-world datasets. The model achieves a high accuracy of 0.9864, not only surpassing the previous results but also providing valuable insights into the reasoning behind the classification. Our model is able to pinpoint specific instructions that lead to malicious behavior in malware samples, aiding human experts in their analysis and driving further advancements in the field. We report our findings and show how causality can be established between malicious code and actual classification by a deep learning model, thus opening up this black-box problem for deeper analysis.