A retargeting method for stereoscopic 3D video

Yi Liu; Lifeng Sun; Shiqiang Yang

doi:10.1007/s41095-015-0016-2

Computational Visual Media 2015, 1(2): 119-127 https://doi.org/10.1007/s41095-015-0016-2

Research Article |

Open Access | Issue | Published: 16 August 2015

A retargeting method for stereoscopic 3D video

Show Author's Information Hide Author's Information Yi Liu^¹(

), Lifeng Sun^², Shiqiang Yang^²

1 Department of Computer Science and Technology, Tsinghua University, Tsinghua NLIST, Beijing 100084, China.

2 Department of Computer Science and Technology, Tsinghua University, Beijing 100084, China.

Keywords:

image warping, stereoscopic 3D video, retargeting, disparity manipulation

Cite this article:

Liu Y, Sun L, Yang S. A retargeting method for stereoscopic 3D video. Computational Visual Media, 2015, 1(2): 119-127. https://doi.org/10.1007/s41095-015-0016-2

Download citation

EndNote(RIS)

BibTeX

658

Views

Downloads

Citations

Crossref

N/A

WoS

Scopus

CSCD

Abstract Full text About this article

Abstract

Abstract We propose a disparity-constrained retargeting method for stereoscopic 3D video, which simultaneously resizes a binocular video to a new aspect ratio and remaps the depth to the perceptual comfort zone. First, we model distortion energies to prevent important video contents from deforming. Then, to maintain depth mapping stability, we model disparity variation energies to constraint the disparity range both in spatial and temporal domains. The last component of our method is a non-uniform, pixel-wise warp to the target resolution based on these energy models. Using this method, we can process the original stereoscopic video to generate new, high-perceptual-quality versions at different display resolutions. For evaluation, we conduct a user study; we also discuss the performance of our method.

Full text

Abstract

Full text

Outline

About this article

A retargeting method for stereoscopic 3D video

Show Author's information Hide Author's Information Yi Liu^¹(

), Lifeng Sun^², Shiqiang Yang^²

1 Department of Computer Science and Technology, Tsinghua University, Tsinghua NLIST, Beijing 100084, China.

2 Department of Computer Science and Technology, Tsinghua University, Beijing 100084, China.

Abstract

Keywords: image warping, stereoscopic 3D video, retargeting, disparity manipulation

References(35)

[1]

Shamir, A.; Sorkine, O. Visual media retargeting. In: ACM SIGGRAPH ASIA 2009 Courses, Article No. 11, 2009.

DOI

[2]

Koppal, S. J.; Zitnick, C. L.; Cohen, M.; Kang, S. B.; Ressler, B.; Colburn, A. A viewer-centric editor for 3D movies. IEEE Computer Graphics and Applications Vol. 31, No. 1, 20-35, 2011.

DOI Google Scholar

[3]

Mendiburu, B. 3D Movie Making: Stereoscopic Digital Cinema from Script to Screen. CRC Press, 2012.

[4]

Kooi, F. L.; Toet, A. Visual comfort of binocular and 3D displays. Displays Vol. 25, Nos. 2–3, 99-108, 2004.

DOI Google Scholar

[5]

Lambooij, M.; Fortuin, M.; Heynderickx, I.; IJsselsteijn, W. Visual discomfort and visual fatigue of stereoscopic displays: A review. Journal of Imaging Science and Technology Vol. 53, No. 3, 30201, 2009.

DOI Google Scholar

[6]

Chen, L.-Q.; Xie, X.; Fan, X.; Ma, W.-Y.; Zhang, H.-J.; Zhou, H.-Q. A visual attention model for adapting images on small displays. Multimedia Systems Vol. 9, No. 4, 353-364, 2003.

DOI Google Scholar

[7]

Avidan, S.; Shamir, A. Seam carving for contentaware image resizing. ACM Transactions on Graphics Vol. 26, No. 3. Article No. 10, 2007.

DOI Google Scholar

[8]

Rubinstein, M.; Shamir, A.; Avidan, S. Improved seam carving for video retargeting. ACM Transactions on Graphics Vol. 27, No. 3, Article No. 16, 2008.

DOI Google Scholar

[9]

Gal, R.; Sorkine, O.; Cohen-Or, D. Feature-aware texturing. In: Proceedings of the 17th Eurographics conference on Rendering Techniques, 297–303, 2006.

[10]

Wolf, L.; Guttmann, M.; Cohen-Or, D. Nonhomogeneous content-driven video-retargeting. In: IEEE 11th International Conference on Computer Vision, 1–6, 2007.

DOI

[11]

Wang, Y.-S.; Tai, C.-L.; Sorkine, O.; Lee, T.-Y. Optimized scale-and-stretch for image resizing. ACM Transactions on Graphics Vol. 27, No. 5, Article No. 118, 2008.

DOI Google Scholar

[12]

Lin, H.-Y.; Chang, C.-C.; Hsieh, C.-H. Cooperative resizing technique for stereo image pairs. In: 2012 International Conference on Software and Computer Applications, Vol. 41, 96–100, 2012.

[13]

Basha, T. D.; Moses, Y.; Avidan, S. Stereo seam carving a geometrically consistent approach. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 35, No. 10, 2513-2525, 2013.

DOI Google Scholar

[14]

Chang, C.-H.; Liang, C.-K.; Chuang, Y.-Y. Contentaware display adaptation and interactive editing for stereoscopic images. IEEE Transactions on Multimedia Vol. 13, No. 4, 589-601, 2011.

DOI Google Scholar

[15]

Niu, Y.; Feng, W.-C.; Liu, F. Enabling warping on stereoscopic images. ACM Transactions on Graphics Vol. 31, No. 6, Article No. 183, 2012.

DOI Google Scholar

[16]

Luo, S.-J.; Shen, I.-C.; Chen, B.-Y.; Cheng, W.-H.; Chuang, Y.-Y. Perspective-aware warping for seamless stereoscopic image cloning. ACM Transactions on Graphics Vol. 31, No. 6, Article No. 182, 2012.

DOI Google Scholar

[17]

Du, S.; Hu, S.; Martin, R. Changing perspective in stereoscopic images. IEEE Transactions on Visualization and Computer Graphics Vol. 19, No. 8, 1288-1297, 2013.

DOI Google Scholar

[18]

Tong, R-F.; Zhang, Y. Cheng, K.-L. StereoPasting: Interactive composition in stereoscopic images. IEEE Transactions on Visualization and Computer Graphics Vol. 19, No. 8, 1375-1385, 2013.

DOI Google Scholar

[19]

Lin, H.-S.; Guan, S.-H.; Lee, C.-T.; Ouhyoung, M. Stereoscopic 3D experience optimization using cropping and warping. In: SIGGRAPH Asia 2011 Sketches, Article No. 40, 2011.

DOI

[20]

Lee, K.-Y.; Chung, C.-D.; Chuang, Y.-Y. Scene warping: Layer-based stereoscopic image resizing. In: IEEE Conference on Computer Vision and Pattern Recognition, 49–56, 2012.

[21]

Lang, M.; Hornung, A.; Wang, O.; Poulakos, S.; Smolic, A.; Gross, M. Nonlinear disparity mapping for stereoscopic 3D. ACM Transactions on Graphics Vol. 29, No. 4, Article No. 75, 2010.

DOI Google Scholar

[22]

Yan, T.; Lau, R. W. H.; Xu, Y.; Huang, L. Depth mapping for stereoscopic videos. International Journal of Computer Vision Vol. 102, Nos. 1–3, 293-307, 2013.

DOI Google Scholar

[23]

Chamaret,

; Boisson,

; Chevance,

Video retargeting for stereoscopic content under 3D viewing constraints. In: Proceedings of SPIE 8288, Stereoscopic Displays and Applications XXIII, 82880H, 2012.10.1117/12.908445

DOI

[24]

Liu, C.-W.; Huang, T.-H.; Chang, M.-H.; Lee, K.-Y.; Liang, C.-K.; Chuang, Y.-Y. 3D cinematography principles and their applications to stereoscopic media processing. In: Proceedings of the 19th ACM international conference on Multimedia, 253–262, 2011.

DOI

[25]

Templin, K.; Didyk, P.; Myszkowski, K.; Hefeeda M. M.; Seidel H.-P.; Matusik, W. Modeling and optimizing eye vergence response to stereoscopic cuts. ACM Transactions on Graphics Vol. 33, No. 4, Article No. 145, 2014.

DOI Google Scholar

[26]

Mu, T.-J.; Sun, J.-J.; Martin, R. R.; Hu, S.-M. A response time model for abrupt changes in binocular disparity. The Visual Computer Vol. 31, No. 5, 675-687, 2014.

DOI Google Scholar

[27]

Harel, J.; Koch, C.; Perona, P. Graph-based visual saliency. In: Advances in Neural Information Processing Systems 19, 545–552, 2006.

[28]

Bay, H.; Tuytelaars, T.; Gool, L. V. SURF: Speeded up robust features. Lecture Notes in Computer Science Vol. 3951, 404-417, 2006.

DOI Google Scholar

[29]

Fischler,

M. A.

; Bolles,

R. C.

Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM Vol. 24, No. 6, 381-395, 1981.10.1145/358669.358692

DOI Google Scholar

[30]

Shi, J.; Tomasi, C. Good features to track. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 593–600, 1994.

[31]

Von Gioi, R. G.; Jakubowicz, J.; Morel, J.-M.; Randall, G. LSD: A line segment detector. Image Processing On Line Vol. 2, 35-55, 2012.

DOI Google Scholar

[32]

Krähenbühl, P.; Lang, M.; Hornung, A.; Gross, M. A system for retargeting of streaming video. ACM Transactions on Graphics Vol. 28, No. 5, Article No. 126, 2009.

DOI Google Scholar

[33]

Paige, C. C.; Saunders, M. A. LSQR: An algorithm for sparse linear equations and sparse least squares. ACM Transactions on Mathematical Software Vol. 8, No. 1, 43-71, 1982.

DOI Google Scholar

[34]

Smolic, A.; Tech, G.; Brust, H. Report on generation of stereo video data base. Mobile3DTV Technical Report, 2010.

Google Scholar

[35]

ITU-T RECOMMENDATION P. Subjective video quality assessment methods for multimedia applications. 1999. Available at https://www.itu.int/rec/T-REC-P.910/en.

About this article

Publication history

Acknowledgements

Rights and permissions

Publication history

Revised: 13 March 2015

Accepted: 11 May 2015

Published: 16 August 2015

Issue date: June 2015

Copyright

Acknowledgements

This work was supported by the National Basic Research Program of China under Grant No. 2011CB302206, the National Natural Science Foundation of China under Grant Nos. 61272226 and 61272231, and Beijing Key Laboratory of Networked Multimedia.

Rights and permissions

This article is published with open access at Springerlink.com

This article is distributed under the terms of the Creative Commons Attribution License which permits any use, distribution, and reproduction in any medium, provided the original author(s) and the source are credited.

Other papers from this open access journal are available at no cost from http://www.springer.com/journal/41095. To submit a manuscript, please go to https://www.editorialmanager.com/cvmj.