Semantic movie summarization based on string of IE-RoleNets

Abstract Roles, their emotion, and interactions between them are three key elements for semantic content understanding of movies. In this paper, we proposed a novel movie summarization method to capture the semantic content in movies based on a string of IE-RoleNets. An IE-RoleNet (interaction and emotion rolenet) models the emotion and interactions of roles in a shot of the movie. The whole movie is represented as a string of IE-RoleNets. Summarization of a movie is transformed into finding an optimal substring with user-specified summarization ratio. Hierarchical substring mining is conducted to find an optimal substring of the whole movie. We have conducted objective and subjective experiments on our method. Experimental results show the ability of our method to capture the semantic content of movies.