As a post-processing technique, re-ranking has demonstrated significant effectiveness in cross-modal retrieval tasks. By mining and processing the information between initial ranking lists, re-ranking process effectively improves retrieval accuracy. The current mainstream cross-modal retrieval re-ranking methods re-rank the initial list based on paired datasets. However, they have poor flexibility because they cannot be easily plugged into existing systems without modifying the original framework and retraining, which makes it difficult to transfer them to other frameworks. Moreover, they cannot be applied in unpaired scenarios. At present, cross-modal retrieval has achieved significant progress by relying on large-scale paired datasets, but it overlooks the problem that labeling such large-scale datasets in practical scenarios requires substantial resources. To address these issues, this paper proposes an unpaired cross-modal retrieval re-ranking method based on neighbor information aggregation. The method improves retrieval performance by mining and utilizing the neighbor information of samples, pushing incorrect answers away from the query input. It searches for local neighbors in the Euclidean neighborhood and for global neighbor expressions through collaborative expression, and then integrates these two types of neighbor information to generate new features for re-calculating semantic similarity with the retrieval input, thus completing a reranking process. Finally, the proposed method is applied as a post-processing technique in several cross-modal retrieval model frameworks and is tested on MSCOCO dataset, with its effectiveness and superiority over other reranking methods being demonstrated.

The existing cross-modal retrieval algorithms based on metric learning ignore the pose differences and domain differences in cross-modal face retrieval tasks. In addition, these algorithms lack learning of global information in the process of metric learning and construct a large number of redundant triplets. Therefore, a cross-modal common representation generation algorithm based on metric learning was proposed in this paper. The algorithm uses the yaw angle equivariant module compensating for yaw angle differences to obtain the image features with robustness, uses the multi-layer attention mechanism to obtain video features with differentiability, uses global triplets and local triplets to jointly train the cross-modal common representation generation network, so as to improve the consistency and accuracy of metric learning. Then it accelerates the convergence of loss functions through the screening of semi-hard triplets. This study proposed a domain adaption algorithm which combines domain calibration and transfer learning to improve the generalization of common representations. The results of comparative experiments on three face video datasets, namely, PB, YTC, and UMD, demonstrate that the algorithm can improve the accuracy of cross-modal face retrieval, and fine-tuning the cross-modal common representation generation network with few samples can improve the accuracy of cross-modal retrieval using target domain images.