Journal Home > Volume 1 , Issue 4

Salient object detection remains one of the most important and active research topics in computer vision, with wide-ranging applications to object recognition, scene understanding, image retrieval, context aware image editing, image compression, etc. Most existing methods directly determine salient objects by exploring various salient object features. Here, we propose a novel graph based ranking method to detect and segment the most salient object in a scene according to its relationship to image border (background) regions, i.e., the background feature. Firstly, we use regions/super-pixels as graph nodes, which are fully connected to enable both long range and short range relations to be modeled. The relationship of each region to the image border (background) is evaluated in two stages: (i) ranking with hard background queries, and (ii) ranking with soft foreground queries. We experimentally show how this two-stage ranking based salient object detection method is complementary to traditional methods, and that integrated results outperform both. Our method allows the exploitation of intrinsic image structure to achieve high quality salient object determination using a quadratic optimization framework, with a closed form solution which can be easily computed. Extensive method evaluation and comparison using three challenging saliency datasets demonstrate that our method consistently outperforms 10 state-of-the-art models by a big margin.


menu
Abstract
Full text
Outline
About this article

SaliencyRank: Two-stage manifold ranking for salient object detection

Show Author's information Wei Qi1Ming-Ming Cheng2Ali Borji3Huchuan Lu4Lian-Fa Bai1( )
Nanjing University of Science and Technology, Nanjing 210094, China.
Nankai University, Tianjin 300353, China.
University of Wisconsin, Milwaukee, WI 53211, USA.
Dalian University of Technology, Dalian 116024, China.

Abstract

Salient object detection remains one of the most important and active research topics in computer vision, with wide-ranging applications to object recognition, scene understanding, image retrieval, context aware image editing, image compression, etc. Most existing methods directly determine salient objects by exploring various salient object features. Here, we propose a novel graph based ranking method to detect and segment the most salient object in a scene according to its relationship to image border (background) regions, i.e., the background feature. Firstly, we use regions/super-pixels as graph nodes, which are fully connected to enable both long range and short range relations to be modeled. The relationship of each region to the image border (background) is evaluated in two stages: (i) ranking with hard background queries, and (ii) ranking with soft foreground queries. We experimentally show how this two-stage ranking based salient object detection method is complementary to traditional methods, and that integrated results outperform both. Our method allows the exploitation of intrinsic image structure to achieve high quality salient object determination using a quadratic optimization framework, with a closed form solution which can be easily computed. Extensive method evaluation and comparison using three challenging saliency datasets demonstrate that our method consistently outperforms 10 state-of-the-art models by a big margin.

Keywords: saliency, salient object detection, visual attention, manifold ranking

References(47)

[1]
Borji, A.; Itti, L. State-of-the-art in visual attention modeling. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 35, No. 1, 185-207, 2013.
[2]
Toet, A. Computational versus psychophysical bottom-up image saliency: A comparative evaluation study. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 33, No. 11, 2131-2146, 2011.
[3]
Itti, L. Automatic foveation for video compression using a neurobiological model of visual attention. IEEE Transactions on Image Processing Vol. 13, No. 10, 1304-1318, 2004.
[4]
Rother, C.; Kolmogorov, V.; Blake, A. “GrabCut”: Interactive foreground extraction using iterated graph cuts. ACM Transactions on Graphics Vol. 23, No.3, 309-314, 2004.
[5]
Sharma, G.; Jurie, F.; Schmid, C. Discriminative spatial saliency for image classification. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 3506-3513, 2012.
DOI
[6]
Avidan, S.; Shamir, A. Seam carving for content-aware image resizing. ACM Transactions on Graphics Vol. 26, No. 3, Article No. 10, 2007.
[7]
Zhang, G.-X.; Cheng, M.-M.; Hu, S.-M.; Martin, R. R. A shape-preserving approach to image resizing. Computer Graphics Forum Vol. 28, No. 7, 1897-1906, 2009.
[8]
Zhang, L.; Huang, H. Hierarchical narrative collage for digital photo album. Computer Graphics Forum Vol. 31, No. 7, 2173-2181, 2012.
[9]
Shen, C.; Zhao, Q. Webpage saliency. In: Lecture Notes in Computer Science, Vol. 8695. Fleet, D.; Pajdla, T.; Schiele, B.; Tuytelaars, T. Eds. Springer International Publishing, 33-46, 2014.
DOI
[10]
Mahadevan, V.; Vasconcelos, N. Saliency-based discriminant tracking. In: Proceeding of IEEE Conference on Computer Vision and Pattern Recognition, 1007-1013, 2009.
DOI
[11]
Shen, X.; Wu, Y. A unified approach to salient object detection via low rank matrix recovery. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 853-860, 2012.
[12]
Yang, C.; Zhang, L.; Lu, H.; Ruan, X.; Yang, M.-H. Saliency detection via graph-based manifold ranking. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 3166-3173, 2013.
DOI
[13]
Li, X.; Lu, H.; Zhang, L.; Ruan, X.; Yang, M.-H. Saliency detection via dense and sparse reconstruction. In: Proceedings of IEEE International Conference on Computer Vision, 2976-2983, 2013.
DOI
[14]
Achanta, R.; Hemami, S.; Estrada, F.; Süsstrunk, S. Frequency-tuned salient region detection. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 1597-1604, 2009.
DOI
[15]
Cheng, M.-M.; Mitra, N. J.; Huang, X.; Torr, P. H. S.; Hu, S.-M. Global contrast based salient region detection.IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 37, No. 3, 569-582, 2015.
[16]
Liu, T.; Yuan, Z.; Sun, J.; Wang, J.; Zheng, N.; Tang, X.; Shum, H.-Y. Learning to detect a salient object. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 33, No. 2, 353-367, 2011.
[17]
Yan, Q.; Xu, L.; Shi, J.; Jia, J. Hierarchical saliency detection. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 1155-1162, 2013.
DOI
[18]
Borji, A.; Sihite, D. N.; Itti, L. Salient object detection: A benchmark. In: Proceedings of the 12th European Conference on Computer Vision, Vol. II, 414-429, 2012.
DOI
[19]
Itti, L.; Koch, C.; Niebur, E. A model of saliency-based visual attention for rapid scene analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 20, No. 11, 1254-1259, 1998.
[20]
Harel, J.; Koch, C.; Perona, P. Graph-based visual saliency. In: Proceedings of Advances in Neural Information Processing Systems 19, 2006. Available at http://papers.nips.cc/paper/3095-graph-based-visual-saliency.pdf.
[21]
Hou, X.; Zhang, L. Saliency detection: A spectral residual approach. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 1-8, 2007.
DOI
[22]
Zhang, L.; Tong, M. H.; Marks, T. K.; Shan, H.; Cottrell, G. W. SUN: A Bayesian framework for saliency using natural statistics. Journal of Vision Vol. 8, No. 7, Article No. 32, 2008.
[23]
Rahtu, E.; Kannala, J.; Salo, M.; Heikkilä, J. Segmenting salient objects from images and videos. In: Proceedings of the 11th European Conference on Computer Vision: Part V, 366-379, 2010.
DOI
[24]
Goferman, S.; Zelnik-Manor, L.; Tal, A. Context-aware saliency detection. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 34, No. 10, 1915-1926, 2012.
[25]
Jiang, B.; Zhang, L.; Lu, H.; Yang, C.; Yang, M.-H. Saliency detection via absorbing Markov chain. In: Proceedings of IEEE International Conference on Computer Vision, 1665-1672, 2013.
DOI
[26]
Jiang, P.; Ling, H.; Yu, J.; Peng, J. Salient region detection by UFO: Uniqueness, focusness and objectness. In: Proceedings of IEEE International Conference on Computer Vision, 1976-1983, 2013.
DOI
[27]
Jiang, H.; Wang, J.; Yuan, Z.; Wu, Y.; Zheng, N.; Li, S. Salient object detection: A discriminative regional feature integration approach. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2083-2090, 2013.
DOI
[28]
Li, X.; Li, Y.; Shen, C.; Dick, A.; Hengel, A. V. D. Contextual hypergraph modeling for salient object detection. In: Proceedings of IEEE International Conference on Computer Vision, 3328-3335, 2013.
DOI
[29]
Zhang, J.; Sclaroff, S. Saliency detection: A Boolean map approach. In: Proceedings of IEEE International Conference on Computer Vision, 153-160, 2013.
DOI
[30]
Kim, J.; Han, D.; Tai, Y.-W.; Kim, J. Salient region detection via high-dimensional color transform. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 883-890, 2014.
DOI
[31]
Jiang, Z.; Davis, L. S. Submodular salient region detection. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2043-2050, 2013.
DOI
[32]
Margolin, R.; Tal, A.; Zelnik-Manor, L. What makes a patch distinct? In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 1139-1146, 2013.
DOI
[33]
Liu, R.; Cao, J.; Lin, Z.; Shan, S. Adaptive partial differential equation learning for visual saliency detection. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 3866-3873, 2014.
DOI
[34]
Li, N.; Ye, J.; Ling, H.; Yu, J. Saliency detection on light field. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2806-2813, 2014.
DOI
[35]
Jiang, M.; Huang, S.; Duan, J.; Zhao, Q. SALICON: Saliency in context. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 1072-1080, 2015.
DOI
[36]
Cheng, M.-M.; Mitra, N. J.; Huang, X.; Hu, S.-M. SalientShape: Group saliency in image collections. The Visual Computer Vol. 30, No. 4, 443-453, 2014.
[37]
Fu, H.; Cao, X.; Tu, Z. Cluster-based co-saliency detection. IEEE Transactions on Image Processing Vol. 22, No. 10, 3766-3778, 2013.
[38]
Wei, Y.; Wen, F.; Zhu, W.; Sun, J. Geodesic saliency using background priors. In: Proceedings of the 12th European Conference on Computer Vision, Vol. III, 29-42, 2012.
DOI
[39]
Zhu, W.; Liang, S.; Wei, Y.; Sun, J. Saliency optimization from robust background detection. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2814-2821, 2014.
DOI
[40]
Zhou, D.; Weston, J.; Gretton, A.; Bousquet, O.; Schölkopf, B. Ranking on data manifolds. In: Proceedings of Advances in Neural Information Processing Systems 16, 2004. Available at http://papers.nips.cc/paper/2447-ranking-on-data-manifolds.pdf.
[41]
Achanta, R.; Shaji, A.; Smith, K.; Lucchi, A.; Fua, P.; Süsstrunk, S. SLIC superpixels compared to state-of-the-art superpixel methods. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 34, No. 11, 2274-2282, 2012.
[42]
Jiang, H.; Wang, J.; Yuan, Z.; Liu, T.; Zheng, N. Automatic salient object segmentation based on context and shape prior. In: Proceedings of the British Machine Vision Conference, 110.1-110.12, 2011.
DOI
[43]
Canny, J. A computational approach to edge detection. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 8, No. 6, 679-698, 1986.
[44]
Krähenbühl, P.; Koltun, V. Efficient inference in fully connected CRFs with Gaussian edge potentials. In: Proceedings of Advances in Neural Information Processing Systems 24, 2011. Available at http://papers.nips.cc/paper/4296-efficient-inference-in-fully-connected-crfs-with-gaussian-edge-potentials.pdf.
[45]
Duan, L.; Wu, C.; Miao, J.; Qing, L.; Fu, Y. Visual saliency detection by spatially weighted dissimilarity. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 473-480, 2011.
DOI
[46]
Perazzi, F.; Krahenbuhl, P.; Pritch, Y.; Hornung, A. Saliency filters: Contrast based filtering for salient region detection. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 733-740, 2012.
DOI
[47]
Cheng, M.-M.; Warrell, J.; Lin, W.-Y.; Zheng, S.; Vineet, V.; Crook, N. Efficient salient region detection with soft image abstraction. In: Proceedings of IEEE International Conference on Computer Vision, 1529-1536, 2013.
DOI
Publication history
Copyright
Acknowledgements
Rights and permissions

Publication history

Revised: 17 October 2015
Accepted: 16 November 2015
Published: 07 January 2016
Issue date: December 2015

Copyright

© The Author(s) 2015

Acknowledgements

L.-F. Bai and M.-M. Cheng were funded by the National Natural Science Foundation of China under project No. 61231014 and No. 61572264, respectively. A. Borji was supported by Defense Advanced Research Projects Agency (No. HR0011-10-C-0034), the National Science Foundation (No. BCS-0827764), and the Army Research Office (No. W911NF-08-1-0360).

Rights and permissions

This article is published with open access at Springerlink.com

This article is distributed under the terms of the Creative Commons Attribution License which permits any use, distribution, and reproduction in any medium, provided the original author(s) and the source are credited.

Other papers from this open access journal are available free of charge from http://www.springer.com/journal/41095. To submit a manuscript, please go to https://www.editorialmanager.com/cvmj.

Return