Journal Home > Volume 4 , Issue 1
Computational Visual Media 2018, 4(1): 3-15 https://doi.org/10.1007/s41095-017-0089-1
Research Article | Open Access | Issue | Published: 15 June 2017

Surface tracking assessment and interaction in texture space

Show Author's Information Johannes Furch1,  Anna Hilsmann1,2,  Peter Eisert1,2( )
Fraunhofer HHI, Berlin, 10587, Germany.
Humboldt University, Berlin, 10099, Germany.
Keywords:
optical flow, surface tracking, assessment, interaction, mesh warp
Cite this article:
Furch J, Hilsmann A, Eisert P. Surface tracking assessment and interaction in texture space. Computational Visual Media, 2018, 4(1): 3-15. https://doi.org/10.1007/s41095-017-0089-1
Download citation
EndNote(RIS)
BibTeX

470

Views

19

Downloads

Citations

3

Crossref

N/A

WoS

1

Scopus

0

CSCD

Abstract Full text Electronic supplementary material About this article

In this paper, we present a novel approach for assessing and interacting with surface tracking algorithms targeting video manipulation in post-production. As tracking inaccuracies are unavoidable, we enable the user to provide small hints to the algorithms instead of correcting erroneous results afterwards. Based on 2D mesh warp-based optical flow estimation, we visualize results and provide tools for user feedback in a consistent reference system, texture space. In this space, accurate tracking results are reflected by static appearance, and errors can easily be spotted as apparent change. A variety of established tools can be utilized to visualize and assess the change between frames. User interaction to improve tracking results becomes more intuitive in texture space, as it can focus on a small region rather than a moving object. We show how established tools can be implemented for interaction in texture space to provide a more intuitive interface allowing more effective and accurate user feedback.


menu
Abstract
Full text
Outline
Electronic supplementary material
About this article

Surface tracking assessment and interaction in texture space

Show Author's information Johannes Furch1, Anna Hilsmann1,2, Peter Eisert1,2( )
Fraunhofer HHI, Berlin, 10587, Germany.
Humboldt University, Berlin, 10099, Germany.

Abstract

In this paper, we present a novel approach for assessing and interacting with surface tracking algorithms targeting video manipulation in post-production. As tracking inaccuracies are unavoidable, we enable the user to provide small hints to the algorithms instead of correcting erroneous results afterwards. Based on 2D mesh warp-based optical flow estimation, we visualize results and provide tools for user feedback in a consistent reference system, texture space. In this space, accurate tracking results are reflected by static appearance, and errors can easily be spotted as apparent change. A variety of established tools can be utilized to visualize and assess the change between frames. User interaction to improve tracking results becomes more intuitive in texture space, as it can focus on a small region rather than a moving object. We show how established tools can be implemented for interaction in texture space to provide a more intuitive interface allowing more effective and accurate user feedback.

Keywords: optical flow, surface tracking, assessment, interaction, mesh warp

References(24)

[1]
Imagineer Systems. mocha Pro. 2016. Available at http://www.imagineersystems.com/products/mocha-pro.
[2]
Foundry. NUKE. 2016. Available at https://www.foundry.com/products/nuke.
[3]
The Pixelfarm. PFTrack. 2016. Available at http://www.thepixelfarm.co.uk/pftrack/.
[4]
Klose, F.; Ruhl, K.; Lipski, C.; Magnor, M. Flowlab-An interactive tool for editing dense image correspondences. In: Proceedings of the Conference for Visual Media Production, 59-66, 2011.
DOI
[5]
Ruhl, K.; Eisemann, M.; Hilsmann, A.; Eisert, P.; Magnor, M. Interactive scene flow editing for improved image-based rendering and virtual spacetime navigation. In: Proceedings of the 23rd ACM International Conference on Multimedia, 631-640, 2015.
DOI
[6]
Zhang, C.; Price, B.; Cohen, S.; Yang, R. Highquality stereo video matching via user interaction and space-time propagation. In: Proceedings of the International Conference on 3D Vision, 71-78, 2013.
DOI
[7]
Re:Vision Effects. Twixtor. 2016. Available at http://revisionfx.com/products/twixtor/.
[8]
Wilkes, L. The role of ocula in stereo post production. Technical Report. The Foundry, 2009.
Google Scholar
[9]
Chrysos, G. G.; Antonakos, E.; Zafeiriou, S.; Snape, P. Offline deformable face tracking in arbitrary videos. In: Proceedings of the IEEE International Conference on Computer Vision Workshops, 1-9, 2015.
DOI
[10]
Rother, C.; Kolmogorov, V.; Blake, A. “GrabCut”: Interactive foreground extraction using iterated graph cuts. ACM Transactions on Graphics Vol. 23, No. 3, 309-314, 2004.
DOI Google Scholar
[11]
Liao, M.; Gao, J.; Yang, R.; Gong, M. Video stereolization: Combining motion analysis with user interaction. IEEE Transactions on Visualization & Computer Graphics Vol. 18, No. 7, 1079-1088, 2012.
DOI Google Scholar
[12]
Wang, O.; Lang, M.; Frei, M.; Hornung, A.; Smolic, A.; Gross, M. Stereobrush: Interactive 2D to 3D conversion using discontinuous warps. In: Proceedings of the 8th Eurographics Symposium on Sketch-Based Interfaces and Modeling, 47-54, 2011.
DOI
[13]
Doron, Y.; Campbell, N. D. F.; Starck, J.; Kautz, J. User directed multi-view-stereo. In: Computer Vision-ACCV 2014 Workshops. Jawahar, C.; Shan, S. Eds. Springer Cham, 299-313, 2014.
DOI
[14]
Bartoli, A.; Zisserman, A. Direct estimation of non-rigid registrations. In: Proceedings of the 15th British Machine Vision Conference, Vol. 2, 899-908, 2004.
DOI
[15]
Zhu, J.; Van Gool, L.; Hoi, S. C. H. Unsupervised face alignment by robust nonrigid mapping. In: Proceedings of the IEEE 12th International Conference on Computer Vision, 1265-1272, 2009.
[16]
Hilsmann, A.; Eisert, P. Joint estimation of deformable motion and photometric parameters in single view videos. In: Proceedings of the IEEE 12th International Conference on Computer Vision Workshops, 390-397, 2009.
DOI
[17]
Gay-Bellile, V.; Bartoli, A.; Sayd, P. Direct estimation of nonrigid registrations with image-based self-occlusion reasoning. IEEE Transactions on Pattern Analysis & Machine Intelligence Vol. 32, No. 1, 87-104, 2010.
DOI Google Scholar
[18]
Seibold, C.; Hilsmann, A.; Eisert, P. Model-based motion blur estimation for the improvement of motion tracking. Computer Vision and Image Understanding , 2017.
DOI Google Scholar
[19]
Hilsmann, A.; Schneider, D. C.; Eisert, P. Image-based tracking of deformable surfaces. In: Object Tracking. InTech, 245-266, 2011.
[20]
Pilet, J.; Lepetit, V.; Fua, P. Real-time nonrigid surface detection. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Vol. 1, 822-828, 2005.
DOI
[21]
Brox, T.; Malik, J. Large displacement optical flow: Descriptor matching in variational motion estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 33, No. 3, 500-513, 2011.
DOI Google Scholar
[22]
Wedel, A.; Cremers, D.; Pock, T.; Bischof, H. Structure- and motion-adaptive regularization for high accuracy optic flow. In: Proceedings of the IEEE 12th International Conference on Computer Vision, 1663-1668, 2009.
DOI
[23]
Brox, T.; Bruhn, A.; Papenberg, N.; Weickert, J. High accuracy optical flow estimation based on a theory for warping. In: Computer Vision-ECCV 2004. Pajdla, T.; Matas, J. Eds. Springer Berlin Heidelberg, 25-36, 2004.
DOI
[24]
Hollywood Camera Work. Face Capture dataset. 2016. Available at https://www.hollywoodcamerawork.com/tracking-plates.html.
File
41095_2017_89_MOESM1_ESM.flv (25.9 MB)
41095_2017_89_MOESM2_ESM.flv (26.6 MB)
41095_2017_89_MOESM3_ESM.flv (41.8 MB)
Publication history
Copyright
Acknowledgements
Rights and permissions

Publication history

Revised: 06 March 2017
Accepted: 29 April 2017
Published: 15 June 2017
Issue date: March 2018

Copyright

© The Author(s) 2017

Acknowledgements

This work was partially funded by the German Science Foundation (Grant No. DFG EI524/2-1) and by the European Commission (Grant Nos. FP7-288238 SCENE and H2020-644629 AutoPost).

Rights and permissions

This article is published with open access at Springerlink.com

The articles published in this journal are distributed under the terms of the Creative Commons Attribution 4.0 International License (http:// creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Other papers from this open access journal are available free of charge from http://www.springer.com/journal/41095. To submit a manuscript, please go to https://www. editorialmanager.com/cvmj.

Return