A Novel Multispectral Vessel Recognition Based on RGB-to-Thermal Image Translation

In the last decade, advances in deep learning have led to considerable progress in the field of ship classification in Red Green Blue (RGB) and Infra-Red (IR) images. However, ship classification performs poorly on images acquired in weak visible light intensity. Multispectral imaging constitutes a potential solution to address such difficulty. In this paper, we first propose Convolutional Neural Network (CNN) for ship classification in multi-spectral images (RGB, IR, etc.). The proposed architectures were trained from scratch and fine-tuned to another pre-trained network. Validation was carried out on the publically available RGB-IR pairs ship dataset VAIS. Unfortunately, owing to the small size of the dataset, the obtained classification result was 59,09%, hence not satisfactory for most applications. We, therefore, proposed a new image data augmentation approach for the generation of IR ship images from RGB images. The generation process was carried out through an adaptation of a Generative Adversarial Network (GAN) network and a Pix2Pix model. In fact, VAIS dataset was kept aside for validation purposes and KAIST RGB-IR pairs dataset was used for the training of our translator. The augmented IR dataset yielded more than a 9% increase in the performance of VAIS IR-based ship classification.