AI Chat Paper
Note: Please note that the following content is generated by AMiner AI. SciOpen does not take any responsibility related to this content.
{{lang === 'zh_CN' ? '文章概述' : 'Summary'}}
{{lang === 'en_US' ? '中' : 'Eng'}}
Chat more with AI
PDF (9.2 MB)
Collect
Submit Manuscript AI Chat Paper
Show Outline
Outline
Show full outline
Hide outline
Outline
Show full outline
Hide outline
Research Article | Open Access

WGI-Net: A weighted group integration network for RGB-D salient object detection

School of Electrical Information Engineering, NortheastPetroleum University, Daqing 163000, China

Yanliang Ge and Cong Zhang contributed equally to this article.

Show Author Information

Abstract

Salient object detection is used as a pre-process in many computer vision tasks (such as salient object segmentation, video salient object detection, etc.). When performing salient object detection, depth information can provide clues to the location of target objects, so effective fusion of RGB and depth feature information is important. In this paper, we propose a new feature information aggregation approach, weighted group integration (WGI), to effectively integrate RGBand depth feature information. We use a dual-branch structure to slice the input RGB image and depth map separately and then merge the results separately by concatenation. As grouped features may lose global information about the target object, we also make use of the idea of residual learning, taking the features captured by the original fusion method as supplementary information to ensure both accuracy and completeness of the fused information. Experiments on five datasets show that our model performs better than typical existing approaches for four evaluation metrics.

References

[1]
Marchesotti, L.; Cifarelli, C.; Csurka, G. A framework for visual saliency detection with applications to image thumbnailing. In: Proceedings of the IEEE 12th International Conference on Computer Vision, 2232-2239, 2009.
[2]
Ding, Y.; Xiao, J.; Yu. J. Importance filtering for image retargeting. In: Proceedings of the Conference on Computer Vision and Pattern Recognition, 89-96, 2011.
[3]
Goferman, S.; Zelnik-Manor, L.; Tal, A. Context-aware saliency detection. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 34, No. 10, 1915-1926, 2012.
[4]
Wang, W.; Shen, J. Deep cropping via attention box prediction and aesthetics assessment. In: Proceedings of the IEEE International Conference on Computer Vision, 2205-2213, 2017.
[5]
Guo, C. L.; Zhang, L. M. A novel multiresolution spatiotemporal saliency detection model and its applications in image and video compression. IEEE Transactions on Image Processing Vol. 19, No. 1, 185-198, 2010.
[6]
Rutishauser, U.; Walther, D.; Koch, C.; Perona, P. Is bottom-up attention useful for object recognition? In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, II, 2004.
[7]
Han, J.; Ngan, K. N.; Li, M. J.; Zhang, H. J. Unsupervised extraction of visual attention objects in color images. IEEE Transactions on Circuits and Systems for Video Technology Vol. 16, No. 1, 141-145, 2006.
[8]
Liu, Z.; Shi, R.; Shen, L. Q.; Xue, Y. Z.; Ngan, K. N., Zhang, Z. Y. Unsupervised salient object segmentation based on kernel density estimation and two-phase graph cut. IEEE Transactions on Multimedia Vol. 14, No. 4, 1275-1289, 2012.
[9]
Jerripothula, K. R.; Cai, J. F.; Yuan, J. S. Image co-segmentation via saliency co-fusion. IEEE Transactions on Multimedia Vol. 18, No. 9, 1896-1909, 2016.
[10]
Ye, L. W.; Liu, Z.; Li, L. N.; Shen, L. Q.; Bai, C.; Wang, Y. Salient object segmentation via effective integration of saliency and objectness. IEEE Transactions on Multimedia Vol. 19, No. 8, 1742-1756, 2017.
[11]
Borji, A.; Frintrop, S.; Sihite, D. N.; Itti, L. Adaptive object tracking by learning background context. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, 23-30, 2012.
[12]
Hong, S.; You, T.; Kwak, S.; Han, B. Online tracking by learning discriminative saliency map with convolutional neural network. In: Proceedings of the International Conference on Machine Learning, 597-606, 2015.
[13]
Lee, H.; Kim, D. Salient region-based online object tracking. In: Proceedings of the IEEE Winter Conference on Applications of Computer Vision, 1170-1177, 2018.
[14]
Gao, Y.; Wang, M.; Tao, D. C.; Ji, R. R.; Dai, Q. H. 3-D object retrieval and recognition with hypergraph analysis. IEEE Transactions on Image Processing Vol. 21, No. 9, 4290-4303, 2012.
[15]
He, J.; Feng, J.; Liu, X.; Cheng, T.; Lin, T.; Chung, H.; Chang, S. Mobile product search with bag of hash bits and boundary reranking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 3005-3012, 2012.
[16]
Hou, Q.; Cheng, M.-M.; Hu, X.; Borji, A.; Tu, Z.; Torr, P. H. Deeply supervised salient object detection with short connections. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 3203-3212, 2017.
[17]
Wang, W.; Zhao, S.; Shen, J.; Hoi, S. C.; Borji, A. Salient object detection with pyramid attention and salient edges. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1448-1457, 2019.
[18]
Fan, D.; Wang, W.; Cheng, M.; Shen, J. Shifting more attention to video salient object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 8554-8564, 2019.
[19]
Bi, H.; Lu, D.; Li, N.; Yang, L.; Guan, H. Multi-level model for video saliency detection. In: Proceedings of the IEEE International Conference on Image Processing, 4654-4658, 2019.
[20]
Fu, H. Z.; Cao, X. C.; Tu, Z. W. Cluster-based co-saliency detection. IEEE Transactions on Image Processing Vol. 22, No. 10, 3766-3778, 2013.
[21]
Zhang, K.; Li, T.; Shen, S.; Liu, B.; Chen, J.; Liu, Q. Adaptive graph convolutional network with attention graph clustering for co-saliency detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 9050-9059, 2020.
[22]
Wang, A. Z.; Wang, M. H.; Li, X. Y.; Mi, Z. T.; Zhou, H. A two-stage Bayesian integration framework for salient object detection on light field. Neural Processing Letters Vol. 46, No. 3, 1083-1094, 2017.
[23]
Zhang, M.; Ji, W.; Piao, Y. R.; Li, J. J.; Zhang, Y.; Xu, S.; Lu, H. LFNet: Light field fusion network for salient object detection. IEEE Transactions on Image Processing Vol. 29, 6276-6287, 2020.
[24]
Zhang, J.; Wang, X. Light field salient object detection via hybrid priors. In: Proceedings of the International Conference on Multimedia Modeling, 361-372, 2020.
[25]
Zhao, J.; Cao, Y.; Fan, D.; Cheng, M.; Li, X.; Zhang, L. Contrast prior and fluid pyramid integration for RGBD salient object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 3927-3936, 2019.
[26]
Zhang, J.; Fan, D. P.; Dai, Y.; Anwar, S.; Saleh, F. S.; Zhang, T.; Barnes, N. UC-Net: Uncertainty inspired RGB-D saliency detection via conditional variational autoencoders. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 8582-8591, 2020.
[27]
Fu, K.; Fan, D. P.; Ji, G. P.; Zhao, Q. JL-DCF: Joint learning and densely-cooperative fusion framework for RGB-D salient object detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 3049-3059, 2020.
[28]
Peng, H. W.; Li, B.; Xiong, W. H.; Hu, W. M.; Ji, R. R. RGBD salient object detection: A benchmark and algorithms. In: Computer Vision - ECCV 2014. Lecture Notes in Computer Science, Vol. 8691. Fleet, D.; Pajdla, T.; Schiele, B.; Tuytelaars, T. Eds. Springer Cham, 92-109, 2014.
[29]
Song, H. K.; Liu, Z.; Du, H.; Sun, G. L.; Le Meur, O.; Ren, T. W. Depth-aware salient object detection and segmentation via multiscale discriminative saliency fusion and bootstrap learning. IEEE Transactions on Image Processing Vol. 26, No. 9, 4204-4216, 2017.
[30]
Qu, L. Q.; He, S. F.; Zhang, J. W.; Tian, J. D.; Tang, Y. D.; Yang, Q. X. RGBD salient object detection via deep fusion. IEEE Transactions on Image Processing Vol. 26, No. 5, 2274-2285, 2017.
[31]
Liu, Z. Y.; Shi, S.; Duan, Q. T.; Zhang, W.; Zhao, P. Salient object detection for RGB-D image by single stream recurrent convolution neural network. Neurocomputing Vol. 363, 46-57, 2019.
[32]
Feng, D.; Barnes, N.; You, S.; Mccarthy, C. Local background enclosure for RGB-D salient object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2343-2350, 2016.
[33]
Shigematsu, R.; Feng, D.; You, S.; Barnes, N. Learning RGB-D salient object detection using background enclosure, depth contrast, and top-down features. In: Proceedings of the IEEE International Conference on Computer Vision Workshops, 2749-2757, 2017.
[34]
Piao, Y.; Ji, W.; Li, J.; Zhang, M.; Lu, H. Depth-induced multi-scale recurrent attention network for saliency detection. In: Proceedings of the IEEE International Conference on Computer Vision, 7254-7263, 2019.
[35]
Chen, H.; Li, Y. F.; Su, D. Multi-modal fusion network with multi-scale multi-path and cross-modal interactions for RGB-D salient object detection. Pattern Recognition Vol. 86, 376-385, 2019.
[36]
Desingh, K.; Madhava Krishna, K.; Rajan, D.; Jawahar, C. V. Depth really matters: Improving visual salient region detection with depth. In: Proceedings of the British Machine Vision Conference, 98.1-98.11, 2013.
[37]
Fan, X.; Liu, Z.; Sun, G. Salient region detection for stereoscopic images. In: Proceedings of the 19th International Conference on Digital Signal Processing, 454-458, 2014.
[38]
Cheng, Y. P.; Fu, H. Z.; Wei, X. X.; Xiao, J. J.; Cao, X. C. Depth enhanced saliency detection method. In: Proceedings of the International Conference on Internet Multimedia Computing and Service, 23-27, 2014.
[39]
Cong, R. M.; Lei, J. J.; Zhang, C. Q.; Huang, Q. M.; Cao, X. C.; Hou, C. P. Saliency detection for stereoscopic images based on depth confidence analysis and multiple cues fusion. IEEE Signal Processing Letters Vol. 23, No. 6, 819-823, 2016.
[40]
Cong, R.; Lei, J.; Fu, H.; Hou, J.; Huang, Q.; Kwong, S. Going from RGB to RGBD saliency: A depth-guided transformation model. IEEE Transactions on Cybernetics Vol. 50, No. 8, 3627-3639, 2019.
[41]
Ju, R.; Ge, L.; Geng, W.; Ren, T.; Wu, G. Depth saliency based on anisotropic center-surround difference. In: Proceedings of the IEEE International Conference on Image Processing, 1115-1119, 2014.
[42]
Quo, J.; Ren, T.; Bei, J. Salient object detection for RGB-D image via saliency evolution. In: Proceedings of the IEEE International Conference on Multimedia and Expo, 1-6, 2016.
[43]
Wang, A. Z.; Wang, M. H. RGB-D salient object detection via minimum barrier distance transform and saliency fusion. IEEE Signal Processing Letters Vol. 24, No. 5, 663-667, 2017.
[44]
Liang, F. F.; Duan, L. J.; Ma, W.; Qiao, Y. H.; Cai, Z.; Qing, L. Y. Stereoscopic saliency model using contrast and depth-guided-background prior. Neurocomputing Vol. 275, 2227-2238, 2018.
[45]
Niu, Y.; Geng, Y.; Li, X.; Liu, F. Leveraging stereopsis for saliency analysis. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 454-461, 2012.
[46]
Chen, H.; Li, Y. F. Progressively complementarity-aware fusion network for RGB-D salient object detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 3051-3060, 2018.
[47]
Han, J. W.; Chen, H.; Liu, N.; Yan, C. G.; Li, X. L. CNNs-based RGB-D saliency detection via cross-view transfer and multiview fusion. IEEE Transactions on Cybernetics Vol. 48, No. 11, 3171-3183, 2018.
[48]
Huang, P.; Shen, C.; Hsiao, H. RGBD salient object detection using spatially coherent deep learning frame-work. In: Proceedings of the IEEE 23rd International Conference on Digital Signal Processing, 1-5, 2018.
[49]
Zhu, C.; Cai, X.; Huang, K.; Li, T. H.; Li, G. PDNet: Prior-model guided depth-enhanced network for salient object detection. In: Proceedings of the IEEE International Conference on Multimedia and Expo, 199-204, 2019.
[50]
Wang, N. N.; Gong, X. J. Adaptive fusion for RGB-D salient object detection. IEEE Access Vol. 7, 55277-55284, 2019.
[51]
Zhou, T.; Fan, D.-P.; Cheng, M.-M.; Shen, J.; Shao, L. RGB-D salient object detection: A survey.Computational Visual Media Vol. 7, No. 1, 37-69, 2021.
[52]
Zhang, J.; Fan, D.-P.; Dai, Y.; Anwar, S.; Saleh, F.; Aliakbarian, S.; Barnes, N. Uncertainty inspired RGB-D saliency detection. arXiv preprint arXiv:2009.03075, 2020.
[53]
Zhang, M.; Zhang, Y.; Piao, Y. R.; Hu, B. Q.; Lu, H. C. Feature reintegration over differential treatment: A top-down and adaptive fusion network for RGB-D salient object detection. In: Proceedings of the 28th ACM International Conference on Multimedia, 4107-4115, 2020.
[54]
Zhang, J.; Wang, M.; Lin, L.; Yang, X.; Gao, J.; Rui, Y. Saliency detection on light field. ACM Transactions on Multimedia Computing, Communications, and Applications Vol. 13, No. 3, 1-22, 2017.
[55]
Chen, H.; Li, Y. F. Three-stream attention-aware network for RGB-D salient object detection. IEEE Transactions on Image Processing Vol. 28, No. 6, 2825-2835, 2019.
[56]
Piao, Y.; Rong, Z.; Zhang, M.; Ren, W.; Lu. H. A2dele: Adaptive and attentive depth distiller for efficient RGB-D salient object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 9060-9069, 2020.
[57]
Li, C. Y.; Cong, R. M.; Kwong, S.; Hou, J. H.; Fu, H. Z.; Zhu, G. P.; Zhang, D, W.; Huang, Q. M. ASIF-net: Attention steered interweave fusion network for RGB-D salient object detection. IEEE Transactions on Cybernetics Vol. 51, No. 1, 88-100, 2021.
[58]
Fan, D. P.; Lin, Z.; Zhang, Z.; Zhu, M. L.; Cheng, M. M. Rethinking RGB-D salient object detection: Models, data sets, and large-scale benchmarks. IEEE Transactions on Neural Networks and Learning Systems , 2020.
[59]
Borji, A.; Sihite, D. N.; Itti, L. Salient object detection: A benchmark. In: Computer Vision - ECCV 2012. Lecture Notes in Computer Science, Vol. 7573. Fitzgibbon, A.; Lazebnik, S.; Perona, P.; Sato, Y.; Schmid, C. Eds. Springer Berlin Heidelberg, 414-429, 2012.
[60]
Achanta, R.; Hemami, S. S.; Estrada, F. J.; Susstrunk, S. Frequency-tuned salient region detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1597-1604, 2009.
[61]
Fan, D.; Cheng, M.; Liu, Y.; Li, T.; Borji, A. Structure-measure: A new way to evaluate foreground maps. In: Proceedings of the IEEE International Conference on Computer Vision, 4558-4567, 2017.
[62]
Fan, D.; Gong, C.; Cao, Y.; Ren, B.; Cheng, M.; Borji, A. Enhanced-alignment measure for binary foreground map evaluation. arXiv preprint arXiv:1805.10421, 2018.
Computational Visual Media
Pages 115-125
Cite this article:
Ge Y, Zhang C, Wang K, et al. WGI-Net: A weighted group integration network for RGB-D salient object detection. Computational Visual Media, 2021, 7(1): 115-125. https://doi.org/10.1007/s41095-020-0200-x

861

Views

20

Downloads

9

Crossref

10

Web of Science

10

Scopus

0

CSCD

Altmetrics

Received: 19 August 2020
Accepted: 19 November 2020
Published: 08 January 2021
© The Author(s) 2020

This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduc-tion in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made.

The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Other papers from this open access journal are available free of charge from http://www.springer.com/journal/41095. To submit a manuscript, please go to https://www. editorialmanager.com/cvmj.

Return