Review Article
Open Access
Issue
Published:
16 May 2022
Open Access
Issue
Published:
16 May 2022
Light field salient object detection: A review and benchmark
Tao Zhou3(
),
),
Cite this article:
Fu K, Jiang Y, Ji G-P, et al.
Light field salient object detection: A review and benchmark.
Computational Visual Media,
2022, 8(4): 509-534.
https://doi.org/10.1007/s41095-021-0256-2
Download citation
1223
Views
107
Downloads
Citations
11
Crossref
9
WoS
14
Scopus
0
CSCD
Salient object detection (SOD) is a long-standing research topic in computer vision with increasing interest in the past decade. Since light fields record comprehensive information of natural scenes that benefit SOD in a number of ways, using light field inputs to improve saliency detection over conventional RGB inputs is an emerging trend. This paper provides the first comprehensive review and a benchmark for light field SOD, which has long been lacking in the saliency community. Firstly, we introduce light fields, including theory and data forms, and then review existing studies on light field SOD, covering ten traditional models, seven deep learning-based models, a comparative study, and a brief review. Existing datasets for light field SOD are also summarized. Secondly, we benchmark nine representative light field SOD models together with several cutting-edge RGB-D SOD models on four widely used light field datasets, providing insightful discussions and analyses, including a comparison between light field SOD and RGB-D SOD models. Due to the inconsistency of current datasets, we further generate complete data and supplement focal stacks, depth maps, and multi-view images for them, making them consistent and uniform. Our supplemental data make a universal benchmark possible. Lastly, light field SOD is a specialised problem, because of its diverse data representations and high dependency on acquisition hardware, so it differs greatly from other saliency detection tasks. We provide nine observations on challenges and future directions, and outline several open issues. All the materials including models, datasets, benchmarking results, and supplemented light field datasets are publicly available at https://github.com/kerenfu/LFSOD-Survey.
menu
Abstract
Full text
Outline
About this article
Light field salient object detection: A review and benchmark
Tao Zhou3(
),
),
Abstract
Salient object detection (SOD) is a long-standing research topic in computer vision with increasing interest in the past decade. Since light fields record comprehensive information of natural scenes that benefit SOD in a number of ways, using light field inputs to improve saliency detection over conventional RGB inputs is an emerging trend. This paper provides the first comprehensive review and a benchmark for light field SOD, which has long been lacking in the saliency community. Firstly, we introduce light fields, including theory and data forms, and then review existing studies on light field SOD, covering ten traditional models, seven deep learning-based models, a comparative study, and a brief review. Existing datasets for light field SOD are also summarized. Secondly, we benchmark nine representative light field SOD models together with several cutting-edge RGB-D SOD models on four widely used light field datasets, providing insightful discussions and analyses, including a comparison between light field SOD and RGB-D SOD models. Due to the inconsistency of current datasets, we further generate complete data and supplement focal stacks, depth maps, and multi-view images for them, making them consistent and uniform. Our supplemental data make a universal benchmark possible. Lastly, light field SOD is a specialised problem, because of its diverse data representations and high dependency on acquisition hardware, so it differs greatly from other saliency detection tasks. We provide nine observations on challenges and future directions, and outline several open issues. All the materials including models, datasets, benchmarking results, and supplemented light field datasets are publicly available at https://github.com/kerenfu/LFSOD-Survey.
Keywords:
deep learning, light field, benchmarking, salient object detection (SOD)
References(108)
[1]
Wang, S. Z.; Liao, W. J.; Surman, P.; Tu, Z. G.; Zheng, Y. J.; Yuan, J. S. Salience guided depth calibration for perceptually optimized compressive light field 3D display. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2031-2040, 2018.
[2]
Cheng, M. M.; Zhang, G. X.; Mitra, N. J.; Huang, X. L.; Hu, S. M. Global contrast based salient region detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 409-416, 2011.
[3]
Borji, A.; Cheng, M. M.; Jiang, H. Z.; Li, J. Salient object detection: A benchmark. IEEE Transactions on Image Processing Vol. 24, No. 12, 5706-5722, 2015.
[4]
Borji, A.; Cheng, M. M.; Hou, Q. B.; Jiang, H. Z.; Li, J. Salient object detection: A survey. Computational Visual Media Vol. 5, No. 2, 117-150, 2019.
[5]
Li, N. Y.; Ye, J. W.; Ji, Y.; Ling, H. B.; Yu, J. Y. Saliency detection on light field. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2806-2813, 2014.
[6]
Li, N. Y.; Ye, J. W.; Ji, Y.; Ling, H. B.; Yu, J. Y. Saliency detection on light field. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 39, No. 8, 1605-1616, 2017.
[7]
Ren, Z. X.; Gao, S. H.; Chia, L. T.; Tsang, I. W. H. Region-based saliency detection and its application in object recognition. IEEE Transactions on Circuits and Systems for Video Technology Vol. 24, No. 5, 769-779, 2014.
[8]
Zhang, D.; Meng, D.; Zhao, L.; Han, J. Bridging saliency detection to weakly supervised object detection based on self-paced curriculum learning. In: Proceedings of the 25th International Joint Conference on Artificial Intelligence, 3538-3544, 2016.
[9]
Rutishauser, U.; Walther, D.; Koch, C.; Perona, P. Is bottom-up attention useful for object recognition? In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, II, 2004.
[10]
Moosmann, F.; Larlus, D.; Jurie, F. Learning saliency maps for object categorization. In: Proceedings of the ECCV’06 Workshop on the Representation and Use of Prior Knowledge in Vision, 2006.
[11]
Cheng, M. M.; Liu, Y.; Lin, W. Y.; Zhang, Z. M.; Rosin, P. L.; Torr, P. H. S. BING: Binarized normed gradients for objectness estimation at 300fps. Computational Visual Media Vol. 5, No. 1, 3-20, 2019.
[12]
Wei, Y. C.; Feng, J. S.; Liang, X. D.; Cheng, M. M.; Zhao, Y.; Yan, S. C. Object region mining with adversarial erasing: A simple classification to semantic segmentation approach. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 6488-6496, 2017.
[13]
Wei, Y. C.; Liang, X. D.; Chen, Y. P.; Shen, X. H.; Cheng, M. M.; Feng, J. S.; Zhao, Y.; Yan, S. STC: A simple to complex framework for weakly-supervised semantic segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 39, No. 11, 2314-2320, 2017.
[14]
Wang, X.; You, S. D.; Li, X.; Ma, H. M. Weakly-supervised semantic segmentation by iteratively mining common object features. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 1354-1362, 2018.
[15]
Wang, W.; Shen, J.; Yang, R.; Porikli, F. Saliency-aware video object segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 40, No. 1, 20-33, 2018.
[16]
Song, H.; Wang, W.; Zhao, S.; Shen, J.; Lam, K.-M. Pyramid dilated deeper ConvLSTM for video salient object detection. In: Computer Vision - ECCV 2018. ECCV 2018. Lecture Notes in Computer Science, Vol. 11215. Ferrari, V.; Hebert, M.; Sminchisescu, C.; Weiss, Y. Eds. Springer Cham, 744-760, 2018.
[17]
Itti, L. Automatic foveation for video compression using a neurobiological model of visual attention. IEEE Transactions on Image Processing Vol. 13, No. 10, 1304-1318, 2004.
[18]
Ma, Y. F.; Hua, X. S.; Lu, L.; Zhang, H. J. A generic framework of user attention model and its application in video summarization. IEEE Transactions on Multimedia Vol. 7, No. 5, 907-919, 2005.
[19]
Ma, Y. F.; Lu, L.; Zhang, H. J.; Li, M. J. A user attention model for video summarization. In: Proceedings of the 10th ACM International Conference on Multimedia, 533-542, 2002.
[20]
Ouerhani, N.; Bracamonte, J.; Hugli, H.; Ansorge, M.; Pellandini, F. Adaptive color image compression based on visual attention. In: Proceedings of the 11th International Conference on Image Analysis and Processing, 416-421, 2001.
[21]
Han, J. G.; Pauwels, E. J.; de Zeeuw, P. Fast saliency-aware multi-modality image fusion. Neurocomputing Vol. 111, 70-80, 2013.
[22]
Jin, S.; Ling, H. B. Scale and object aware image retargeting for thumbnail browsing. In: Proceedings of the International Conference on Computer Vision, 1511-1518, 2011.
[23]
Sugano, Y.; Matsushita, Y.; Sato, Y. Calibration-free gaze sensing using saliency maps. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2667-2674, 2010.
[24]
Borji, A.; Itti, L. Defending Yarbus: Eye movements reveal observers’ task. Journal of Vision Vol. 14, No. 3, 29, 2014.
[25]
Fu, K. R.; Zhao, Q. J.; Yu-Hua Gu, I.; Yang, J. Deepside: A general deep framework for salient object detection. Neurocomputing Vol. 356, 69-82, 2019.
[26]
Wang, W. G.; Shen, J. B.; Shao, L.; Porikli, F. Correspondence driven saliency transfer. IEEE Transactions on Image Processing Vol. 25, No. 11, 5025-5034, 2016.
[27]
Zhang, P. P.; Wang, D.; Lu, H. C.; Wang, H. Y.; Ruan, X. Amulet: Aggregating multi-level convolutional features for salient object detection. In: Proceedings of the IEEE International Conference on Computer Vision, 202-211, 2017.
[28]
Feng, M. Y.; Lu, H. C.; Ding, E. R. Attentive feedback network for boundary-aware salient object detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 1623-1632, 2019.
[29]
Zhang, P. P.; Wang, D.; Lu, H. C.; Wang, H. Y.; Yin, B. C. Learning uncertain convolutional features for accurate saliency detection. In: Proceedings of the IEEE International Conference on Computer Vision, 212-221, 2017.
[30]
Zhang, J.; Wang, M.; Gao, J.; Wang, Y.; Zhang, X.; Wu, X. Saliency detection with a deeper investigation of light field. In: Proceedings of the 24th International Conference on Artificial Intelligence, 2212-2218, 2015.
[31]
Zhang, M.; Li, J.; Ji, W.; Piao, Y.; Lu, H. Memory-oriented decoder for light field salient object detection. In: Proceedings of the 33rd Conference on Neural Information Processing Systems, 898-908, 2019.
[32]
Piao, Y. R.; Rong, Z. K.; Zhang, M.; Lu, H. C. Exploit and replace: An asymmetrical two-stream architecture for versatile light field saliency detection. Proceedings of the AAAI Conference on Artificial Intelligence Vol. 34, No. 7, 11865-11873, 2020.
[33]
Fu, K. R.; Fan, D. P.; Ji, G. P.; Zhao, Q. J. JL-DCF: Joint learning and densely-cooperative fusion framework for RGB-D salient object detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 3049-3059, 2020.
[34]
Fu, K. R.; Fan, D. P.; Ji, G. P.; Zhao, Q. J.; Shen, J. B.; Zhu, C. Siamese network for RGB-D salient object detection and beyond. IEEE Transactions on Pattern Analysis and Machine Intelligence , 2021.
[35]
Fan, D. P.; Zhai, Y.; Borji, A.; Yang, J.; Shao, L. BBS-Net: RGB-D salient object detection with a bifurcated backbone strategy network. In: Computer Vision - ECCV 2020. Lecture Notes in Computer Science, Vol. 12357. Vedaldi, A.; Bischof, H.; Brox, T.; Frahm, J. M. Eds. Springer Cham, 275-292, 2020.
[36]
Zhang, M.; Fei, S. X.; Liu, J.; Xu, S.; Piao, Y. R.; Lu, H. C. Asymmetric two-stream architecture for accurate RGB-D saliency detection. In: Computer Vision - ECCV 2020. Lecture Notes in Computer Science, Vol. 12373. Vedaldi, A.; Bischof, H.; Brox, T.; Frahm, J. M. Eds. Springer Cham, 374-390, 2020.
[37]
Gershun, A. The light field. Studies in Applied Mathematics Vol. 18, Nos. 1-4, 51-151, 1939.
[38]
Jeon, H. G.; Park, J.; Choe, G.; Park, J.; Bok, Y.; Tai, Y. W.; Kweon, I.-S. Accurate depth map estimation from a lenslet light field camera. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1547-1555, 2015.
[39]
Tao, M. W.; Hadap, S.; Malik, J.; Ramamoorthi, R. Depth from combining defocus and correspondence using light-field cameras. In: Proceedings of the IEEE International Conference on Computer Vision, 673-680, 2013.
[40]
Tao, M. W.; Srinivasan, P. P.; Malik, J.; Rusinkiewicz, S.; Ramamoorthi, R. Depth from shading, defocus, and correspondence using light-field angular coherence. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1940-1948, 2015.
[41]
Wang, T. C.; Efros, A. A.; Ramamoorthi, R. Occlusion-aware depth estimation using light-field cameras. In: Proceedings of the IEEE International Conference on Computer Vision, 3487-3495, 2015.
[42]
Ng, R.; Levoy, M.; Brédif, M.; Duval, G.; Horowitz, M.; Hanrahan, P. Light field photography with a hand-held plenopic camera. Stanford Tech Report CTSR 2005-02, 2005.
[43]
Piao, Y.; Zhang, Y.; Zhang, M.; Ji, X. Dynamic fusion network for light field depth estimation. arXiv preprint arXiv:2104.05969, 2021.
[44]
Piao, Y.; Ji, X.; Zhang, M.; Zhang, Y. Learning multi-modal information for robust light field depth estimation. arXiv preprint arXiv:2104.05971, 2021.
[45]
Piao, Y.; Rong, Z.; Zhang, M.; Li, X.; Lu, H. Deep light-field-driven saliency detection from a single view. In: Proceedings of the 28th International Joint Conference on Artificial Intelligence, 904-911, 2019.
[46]
Jiang, P.; Ling, H. B.; Yu, J. Y.; Peng, J. L. Salient region detection by UFO: Uniqueness, focusness and objectness. In: Proceedings of the IEEE International Conference on Computer Vision, 1976-1983, 2013.
[47]
Buehler, C.; Bosse, M.; McMillan, L.; Gortler, S.; Cohen, M. Unstructured lumigraph rendering. In: Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques, 425-432, 2001.
[48]
Zhang, X. D.; Wang, Y.; Zhang, J.; Hu, L. M.; Wang, M. Light field saliency vs. 2D saliency: A comparative study. Neurocomputing Vol. 166, 389-396, 2015.
[49]
Zhou, T.; Fan, D. P.; Cheng, M. M.; Shen, J. B.; Shao, L. RGB-D salient object detection: A survey.Computational Visual Media Vol. 7, No. 1, 37-69, 2021.
[50]
Li, N. Y.; Sun, B. L.; Yu, J. Y. A weighted sparse coding framework for saliency detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 5216-5223, 2015.
[51]
Sheng, H.; Zhang, S.; Liu, X. Y.; Xiong, Z. Relative location for light field saliency detection. In: Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, 1631-1635, 2016.
[52]
Wang, A. Z.; Wang, M. H.; Li, X. Y.; Mi, Z. T.; Zhou, H. A two-stage Bayesian integration framework for salient object detection on light field. Neural Processing Letters Vol. 46, No. 3, 1083-1094, 2017.
[53]
Zhang, J.; Wang, M.; Lin, L.; Yang, X.; Gao, J.; Rui, Y. Saliency detection on light field. ACM Transactions on Multimedia Computing, Communications, and Applications Vol. 13, No. 3, Article No. 32, 2017.
[54]
Wang, X.; Dong, Y. Y.; Zhang, Q.; Wang, Q. Region-based depth feature descriptor for saliency detection on light field. Multimedia Tools and Applications Vol. 80, No. 11, 16329-16346, 2021.
[55]
Piao, Y. R.; Li, X.; Zhang, M.; Yu, J. Y.; Lu, H. C. Saliency detection via depth-induced cellular automata on light field. IEEE Transactions on Image Processing Vol. 29, 1879-1889, 2020.
[56]
Wang, H. Q.; Yan, B.; Wang, X. Z.; Zhang, Y. B.; Yang, Y. Accurate saliency detection based on depth feature of 3D images. Multimedia Tools and Applications Vol. 77, No. 12, 14655-14672, 2018.
[57]
Wang, T. T.; Piao, Y. R.; Lu, H. C.; Li, X.; Zhang, L. H. Deep learning for light field saliency detection. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, 8837-8847, 2019.
[58]
Zhang, M.; Ji, W.; Piao, Y. R.; Li, J. J.; Zhang, Y.; Xu, S.; Lu, H. LFNet: Light field fusion network for salient object detection. IEEE Transactions on Image Processing Vol. 29, 6276-6287, 2020.
[59]
Zhang, J.; Liu, Y. M.; Zhang, S. P.; Poppe, R.; Wang, M. Light field saliency detection with deep convolutional networks. IEEE Transactions on Image Processing Vol. 29, 4421-4434, 2020.
[60]
Zhang, Q. D.; Wang, S. Q.; Wang, X.; Sun, Z. H.; Kwong, S.; Jiang, J. M. A multi-task collaborative network for light field salient object detection. IEEE Transactions on Circuits and Systems for Video Technology Vol. 31, No. 5, 1849-1861, 2021.
[61]
Zhang, M.; Ren, W. S.; Piao, Y. R.; Rong, Z. K.; Lu, H. C. Select, supplement and focus for RGB-D saliency detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 3469-3478, 2020.
[62]
Zhang, J.; Fan, D. P.; Dai, Y. C.; Anwar, S.; Saleh, F. S.; Zhang, T.; Barnes, N. UC-net: Uncertainty inspired RGB-D saliency detection via conditional variational autoencoders. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 8579-8588, 2020.
[63]
Fan, D. P.; Lin, Z.; Zhang, Z.; Zhu, M. L.; Cheng, M. M. Rethinking RGB-D salient object detection: Models, data sets, and large-scale benchmarks. IEEE Transactions on Neural Networks and Learning Systems Vol. 32, No. 5, 2075-2089, 2021.
[64]
Liu, N.; Zhang, N.; Han, J. W. Learning selective self-mutual attention for RGB-D saliency detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 13753-13762, 2020.
[65]
Li, C. Y.; Cong, R. M.; Piao, Y. R.; Xu, Q. Q.; Loy, C. C. RGB-D salient object detection with cross-modality modulation and selection. In: Computer Vision - ECCV 2020. Lecture Notes in Computer Science, Vol. 12353. Vedaldi, A.; Bischof, H.; Brox, T.; Frahm, J. M. Eds. Springer Cham, 225-241, 2020.
[66]
Pang, Y.; Zhang, L.; Zhao, X.; Lu, H. Hierarchical dynamic filtering network for RGB-D salient object detection. In: Computer Vision - ECCV 2020. Lecture Notes in Computer Science, Vol. 12370. Vedaldi, A.; Bischof, H.; Brox, T.; Frahm, J. M. Eds. Springer Cham, 235-252, 2020.
[67]
Adelson, E.; Bergen, J. The Plenoptic function and the elements of early vision. In: Computational Models of Visual Processing. MIT Press, 3-20, 1991.
[68]
Levoy, M.; Hanrahan, P. Light field rendering. In: Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques, 31-42, 1996.
[69]
Agarwala, A.; Dontcheva, M.; Agrawala, M.; Drucker, S.; Colburn, A.; Curless, B.; Salesin, D.; Cohen, M. Interactive digital photomontage. ACM Transactions on Graphics Vol. 23, No. 3, 294-302, 2004.
[70]
Kuthirummal, S.; Nagahara, H.; Zhou, C. Y.; Nayar, S. K. Flexible depth of field photography. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 33, No. 1, 58-71, 2011.
[71]
Zhu, W. J.; Liang, S.; Wei, Y. C.; Sun, J. Saliency optimization from robust background detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2814-2821, 2014.
[72]
He, K. M.; Sun, J.; Tang, X. O. Single image haze removal using dark channel prior. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 33, No. 12, 2341-2353, 2011.
[73]
Shelhamer, E.; Long, J.; Darrell, T. Fully convolutional networks for semantic segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 39, No. 4, 640-651, 2017.
[74]
Shi, X.; Chen, Z.; Wang, H.; Yeung, D.; Wong, W.; Woo, W. Convolutional LSTM network: A machine learning approach for precipitation nowcasting. In: Proceedings of the 28th International Conference on Neural Information Processing Systems, 802-810, 2015.
[75]
Chen, L. C.; Papandreou, G.; Kokkinos, I.; Murphy, K.; Yuille, A. L. DeepLab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 40, No. 4, 834-848, 2018.
[76]
Yang, C.; Zhang, L. H.; Lu, H. C.; Ruan, X.; Yang, M. H. Saliency detection via graph-based manifold ranking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 3166-3173, 2013.
[77]
Wang, L. J.; Lu, H. C.; Wang, Y. F.; Feng, M. Y.; Wang, D.; Yin, B. C.; Ruan, X. Learning to detect salient objects with image-level supervision. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 3796-3805, 2017.
[78]
Achanta, R.; Hemami, S.; Estrada, F.; Susstrunk, S. Frequency-tuned salient region detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1597-1604, 2009.
[79]
Perazzi, F.; Krähenbühl, P.; Pritch, Y.; Hornung, A. Saliency filters: Contrast based filtering for salient region detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 733-740, 2012.
[80]
Fan, D. P.; Cheng, M. M.; Liu, Y.; Li, T.; Borji, A. Structure-measure: A new way to evaluate foreground maps. In: Proceedings of the IEEE International Conference on Computer Vision, 4558-4567, 2017.
[81]
Zhao, J. X.; Cao, Y.; Fan, D. P.; Cheng, M. M.; Li, X. Y.; Zhang, L. Contrast prior and fluid pyramid integration for RGBD salient object detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 3922-3931, 2019.
[82]
Fan, D. P.; Gong, C.; Cao, Y.; Ren, B.; Cheng, M. M.; Borji, A. Enhanced-alignment measure for binary foreground map evaluation. In: Proceedings of the 27th International Joint Conference on Artificial Intelligence, 698-704, 2018.
[83]
Wu, Z.; Su, L.; Huang, Q. M. Stacked cross refinement network for edge-aware salient object detection. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, 7263-7272, 2019.
[84]
Qin, X. B.; Zhang, Z. C.; Huang, C. Y.; Gao, C.; Dehghan, M.; Jagersand, M. BASNet: Boundary-aware salient object detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 7471-7481, 2019.
[85]
Li, G. Y.; Liu, Z.; Ling, H. B. ICNet: Information conversion network for RGB-D based salient object detection. IEEE Transactions on Image Processing Vol. 29, 4873-4884, 2020.
[86]
Tsiami, A.; Koutras, P.; Maragos, P. STAViS: Spatio-temporal AudioVisual saliency network. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 4765-4775, 2020.
[87]
Fan, D. P.; Wang, W. G.; Cheng, M. M.; Shen, J. B. Shifting more attention to video salient object detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 8546-8556, 2019.
[88]
Zhao, W. D.; Zhao, F.; Wang, D.; Lu, H. C. Defocus blur detection via multi-stream bottom-top-bottom network. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 42, No. 8, 1884-1897, 2020.
[89]
Park, J.; Tai, Y. W.; Cho, D.; Kweon, I. S. A unified approach of multi-scale deep and hand-crafted features for defocus estimation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2760-2769, 2017.
[90]
Zeng, Y.; Zhuge, Y. Z.; Lu, H. C.; Zhang, L. H.; Qian, M. Y.; Yu, Y. Z. Multi-source weak supervision for saliency detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 6067-6076, 2019.
[91]
Qian, M. Y.; Qi, J. Q.; Zhang, L. H.; Feng, M. Y.; Lu, H. C. Language-aware weak supervision for salient object detection. Pattern Recognition Vol. 96, 106955, 2019.
[92]
Chen, T. L.; Liu, S. J.; Chang, S. Y.; Cheng, Y.; Amini, L.; Wang, Z. Y. Adversarial robustness: From self-supervised pre-training to fine-tuning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 696-705, 2020.
[93]
Dai, A.; Diller, C.; Niessner, M. SG-NN: Sparse generative neural networks for self-supervised scene completion of RGB-D scans. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 846-855, 2020.
[94]
Zeng, Y.; Zhang, P. P.; Lin, Z.; Zhang, J. M.; Lu, H. C. Towards high-resolution salient object detection. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, 7233-7242, 2019.
[95]
Cai, Z. W.; Vasconcelos, N. Cascade R-CNN: High quality object detection and instance segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 43, No. 5, 1483-1498, 2021.
[96]
Chen, K.; Pang, J. M.; Wang, J. Q.; Xiong, Y.; Li, X. X.; Sun, S. Y.; Feng, W.; Liu, Z.; Shi, J.; Ouyang, W.; et al. Hybrid task cascade for instance segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 4969-4978, 2019.
[97]
Liu, S.; Qi, L.; Qin, H. F.; Shi, J. P.; Jia, J. Y. Path aggregation network for instance segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 8759-8768, 2018.
[98]
Li, G.; Xie, Y.; Lin, L.; Yu, Y. Instance-level salient object segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 247-256, 2017.
[99]
Fan, R. C.; Cheng, M. M.; Hou, Q. B.; Mu, T. J.; Wang, J. D.; Hu, S. M. S4Net: Single stage salient-instance segmentation. Computational Visual Media Vol. 6, No. 2, 191-204, 2020.
[100]
Karpathy, A.; Li, F. F. Deep visual-semantic alignments for generating image descriptions. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 39, No. 4, 664-676, 2017.
[101]
Wei, Y.; Xia, W.; Huang, J.; Ni, B.; Dong, J.; Zhao, Y.; Yan, S. CNN: Single-label to multi-label. arXiv preprint arXiv:1406.5726, 2014.
[102]
Chen, X. L.; Gupta, A. Webly supervised learning of convolutional networks. In: Proceedings of the IEEE International Conference on Computer Vision, 1431-1439, 2015.
[103]
Lai, B. S.; Gong, X. J. Saliency guided dictionary learning for weakly-supervised image parsing. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 3630-3639, 2016.
[104]
Tian, X.; Xu, K.; Yang, X.; Yin, B.; Lau, R. Weakly-supervised salient instance detection. In: Proceedings of the Conference on British Machine Vision Conference, 2020.
[105]
Borji, A. Saliency prediction in the deep learning era: Successes and limitations. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 43, No. 2, 679-700, 2021.
[106]
Borji, A.; Itti, L. State-of-the-art in visual attention modeling. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 35, No. 1, 185-207, 2013.
[107]
Fan, D. P.; Ji, G. P.; Sun, G. L.; Cheng, M. M.; Shen, J. B.; Shao, L. Camouflaged object detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2774-2784, 2020.
[108]
Xu, Y. C.; Nagahara, H.; Shimada, A.; Taniguchi, R. I. TransCut: Transparent object segmentation from a light-field image. In: Proceedings of the IEEE International Conference on Computer Vision, 3442-3450, 2015.
Publication history
Copyright
Acknowledgements
Rights and permissions
Publication history
Received: 23 July 2021
Accepted: 03 October 2021
Published:
16 May 2022
Issue date: December 2022
Copyright
© The Author(s) 2022.
Acknowledgements
Keren Fu was supported by the National Natural Science Foundation of China (Nos. 62176169 and61703077) and SCU-Luzhou Municipal People’s Government Strategic Cooperation Project (No. 2020CDLZ-10). Tao Zhou was supported by the National Natural Science Foundation of China (No.62172228). Qijun Zhao was supported by the National Natural Science Foundation of China (No. 61773270).
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduc-tion in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made.
The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.
To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
Other papers from this open access journal are available free of charge from http://www.springer.com/journal/41095. To submit a manuscript, please go to https://www. editorialmanager.com/cvmj.
