Journal Home > Volume 1 , Issue 1
Computational Visual Media 2015, 1(1): 49-57 https://doi.org/10.1007/s41095-015-0008-2
Research Article | Open Access | Issue | Published: 08 August 2015

Panorama completion for street views

Show Author's Information Zhe Zhu1,  Ralph R. Martin2,  Shi-Min Hu1( )
TNLIST, Department of Computer Science and Technology, Tsinghua University, Beijing 100084, China.
School of Computer Science & Informatics, Cardiff University, UK.
Keywords:
image completion, panorama, street views, structure-rectifying warp
Cite this article:
Zhu Z, Martin RR, Hu S-M. Panorama completion for street views. Computational Visual Media, 2015, 1(1): 49-57. https://doi.org/10.1007/s41095-015-0008-2
Download citation
EndNote(RIS)
BibTeX

596

Views

20

Downloads

Citations

12

Crossref

N/A

WoS

14

Scopus

0

CSCD

Abstract Full text About this article

This paper considers panorama images used for street views. Their viewing angle of 360 ∘ causes pixels at the top and bottom to appear stretched and warped. Although current image completion algorithms work well, they cannot be directly used in the presence of such distortions found in panoramas of street views. We thus propose a novel approach to complete such 360 ∘ panoramas using optimization-based projection to deal with distortions. Experimental results show that our approach is efficient and provides an improvement over standard image completion algorithms.


menu
Abstract
Full text
Outline
About this article

Panorama completion for street views

Show Author's information Zhe Zhu1, Ralph R. Martin2, Shi-Min Hu1( )
TNLIST, Department of Computer Science and Technology, Tsinghua University, Beijing 100084, China.
School of Computer Science & Informatics, Cardiff University, UK.

Abstract

This paper considers panorama images used for street views. Their viewing angle of 360 ∘ causes pixels at the top and bottom to appear stretched and warped. Although current image completion algorithms work well, they cannot be directly used in the presence of such distortions found in panoramas of street views. We thus propose a novel approach to complete such 360 ∘ panoramas using optimization-based projection to deal with distortions. Experimental results show that our approach is efficient and provides an improvement over standard image completion algorithms.

Keywords: image completion, panorama, street views, structure-rectifying warp

References(25)

[1]
Brown, M.; Lowe, D. G. Recognising panoramas. In: Proceedings of the Ninth IEEE International Conference on Computer Vision, Vol. 2, 1218-1225, 2003.
DOI
[2]
Brown, M.; Lowe, D. G. Automatic panoramic image stitching using invariant features. International Journal of Computer Vision Vol. 74, No. 1, 59-73, 2007.
DOI Google Scholar
[3]
Summa, B.; Tierny, J.; Pascucci, V. Panorama weaving: Fast and flexible seam processing. ACM Transactions on Graphics Vol. 31, No. 4, Article No. 83, 2012.
DOI Google Scholar
[4]
Gao, J.; Brown, M. S. An interactive editing tool for correcting panoramas. In: SIGGRAPH Asia 2012 Technical Briefs, Article No. 31, 2012.
DOI
[5]
Ballester, C.; Bertalmio, M.; Caselles, V.; Sapiro, G.; Verdera, J. Filling-in by joint interpolation of vector fields and gray levels. IEEE Transactions on Image Processing Vol. 10, No. 8, 1200-1211, 2001.
DOI Google Scholar
[6]
Bertalmio, M.; Sapiro, G.; Caselles, V.; Ballester, C. Image inpainting. In: Proceedings of the 27th annual conference on Computer graphics and interactive techniques, 417-424, 2000.
DOI
[7]
Bertalmio, M.; Vese, L.; Sapiro, G.; Osher, S. Simultaneous structure and texture image inpainting. In: Proceedings 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Vol. 2, II-707-12, 2003.
DOI
[8]
Levin, A.; Zomet, A.; Weiss, Y. Learning how to inpaint from global image statistics. In: Proceedings of the Ninth IEEE International Conference on Computer Vision, Vol. 2, 305, 2003.
DOI
[9]
Roth, S.; Black, M. J. Fields of experts: A framework for learning image priors. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Vol. 2, 860-867, 2005.
[10]
Wexler, Y.; Shechtman, E.; Irani, M. Space-time completion of video. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 29, No. 3, 463-476, 2007.
DOI Google Scholar
[11]
Barnes, C.; Shechtman, E.; Finkelstein, A.; Goldman, D. Patchmatch: A randomized correspondence algorithm for structural image editing. ACM Transactions on Graphics Vol. 28, No. 3, Article No. 24, 2009.
DOI Google Scholar
[12]
Darabi, S.; Shechtman, E.; Barnes, C.; Goldman, D. B.; Sen, P. Image melding: Combining inconsistent images using patch-based synthesis. ACM Transactions on Graphics Vol. 31, No. 4, Article No. 82, 2012.
DOI Google Scholar
[13]
He, K.; Sun, J. Image completion approaches using the statistics of similar patches. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 36, No. 12, 2423-2435, 2014.
DOI Google Scholar
[14]
Information on http://www.adobe.com/technology/ projects.html.
DOI
[15]
Sun, J.; Yuan, L.; Jia, J.; Shum, H.-Y. Image completion with structure propagation. ACM Transactions on Graphics Vol. 24, No. 3, 861-868, 2005.
DOI Google Scholar
[16]
Huang, H.; Yin, K.; Gong, M.; Lischinski, D.; Cohen-Or, D.; Ascher, U.; Chen, B. "Mind the gap": Tele-registration for structure-driven image completion. ACM Transactions on Graphics Vol. 32, No. 6, Article No. 174, 2013.
DOI Google Scholar
[17]
Huang, J.-B.; Kang, S. B.; Ahuja, N.; Kopf, J. Image completion using planar structure guidance. ACM Transactions on Graphics Vol. 33, No. 4, Article No. 129, 2014.
DOI Google Scholar
[18]
Zontak, M.; Irani, M. Internal statistics of a single natural image. In: Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition, 977-984, 2011.
DOI
[19]
Carroll, R.; Agrawal, M.; Agarwala, A. Optimizing content-preserving projections for wide-angle images. ACM Transactions on Graphics Vol. 28, No. 3, Article No. 43, 2009.
DOI Google Scholar
[20]
Von Gioi, R. G.; Jakubowicz, J.; Morel, J.-M.; Randall, G. LSD: A fast line segment detector with a false detection control. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 32, No. 4, 722-732, 2010.
Google Scholar
[21]
Chang, C.-H.; Chuang, Y.-Y. A line-structure-preserving approach to image resizing. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition, 1075-1082, 2012.
[22]
Zhang, G.-X.; Cheng, M.-M.; Hu, S.-M.; Martin, R. R. A shape-preserving approach to image resizing. Computer Graphics Forum Vol. 28, No. 7, 1897-1906, 2009.
DOI Google Scholar
[23]
Hel-Or, Y.; Hel-Or, H. Real-time pattern matching using projection kernels. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 27, No. 9, 1430-1445, 2005.
DOI Google Scholar
[24]
Sun, J. Computing nearest-neighbor fields via propagation-assisted KD-trees. In: Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition, 111-118, 2012.
[25]
Simakov, D.; Caspi, Y.; Shechtman, E.; Irani, M. Summarizing visual data using bidirectional similarity. In: IEEE Conference on Computer Vision and Pattern Recognition 2008. Available at http://www.wisdom. weizmann.ac.il/∼vision/VisualSummary/bidirectional_ similarity_CVPR2008.pdf.
Publication history
Copyright
Acknowledgements
Rights and permissions

Publication history

Revised: 31 October 2014
Accepted: 16 February 2015
Published: 08 August 2015
Issue date: March 2015

Copyright

© The Author(s) 2015

Acknowledgements

This work was supported by the National Basic Research Program of China (No. 2011CB302205), the National Natural Science Foundation of China (No. 61120106007), the National High-tech R&D Program of China (No. 2012AA011802), and EPSRC Travel Grant, a research grant of Beijing Higher Institution Engineering Research Center, and Tsinghua University Initiative Scientific Research Program.

Rights and permissions

This article is published with open access at Springerlink.com

This article is distributed under the terms of the Creative Commons Attribution License which permits any use, distribution, and reproduction in any medium, provided the original author(s) and the source are credited.

Return