Journal Home > Volume 2 , Issue 1
Computational Visual Media 2016, 2(1): 87-96 https://doi.org/10.1007/s41095-016-0032-x
Research Article | Open Access | Issue | Published: 29 January 2016

An interactive approach for functional prototype recovery from a single RGBD image

Show Author's Information Yuliang Rong1,  Youyi Zheng2,  Tianjia Shao1( ),  Yin Yang3,  Kun Zhou1
State Key Lab of CAD&CG, Zhejiang University, Hangzhou 310058, China.
ShanghaiTech University, Shanghai 200031, China.
The University of New Mexico, Albuquerque, NM 87131, USA.
Keywords:
prototype, functionality, cuboid proxy, part relations, shape analysis
Cite this article:
Rong Y, Zheng Y, Shao T, et al. An interactive approach for functional prototype recovery from a single RGBD image. Computational Visual Media, 2016, 2(1): 87-96. https://doi.org/10.1007/s41095-016-0032-x
Download citation
EndNote(RIS)
BibTeX

590

Views

28

Downloads

Citations

5

Crossref

N/A

WoS

6

Scopus

0

CSCD

Abstract Full text Electronic supplementary material About this article

Inferring the functionality of an object from a single RGBD image is difficult for two reasons: lack of semantic information about the object, and missing data due to occlusion. In this paper, we present an interactive framework to recover a 3D functional prototype from a single RGBD image. Instead of precisely reconstructing the object geometry for the prototype, we mainly focus on recovering the object’s functionality along with its geometry. Our system allows users to scribble on the image to create initial rough proxies for the parts. After user annotation of high-level relations between parts, our system automatically jointly optimizes detailed joint parameters (axis and position) and part geometry parameters (size, orientation, and position). Such prototype recovery enables a better understanding of the underlying image geometry and allows for further physically plausible manipulation. We demonstrate our framework on various indoor objects with simple or hybrid functions.


menu
Abstract
Full text
Outline
Electronic supplementary material
About this article

An interactive approach for functional prototype recovery from a single RGBD image

Show Author's information Yuliang Rong1, Youyi Zheng2, Tianjia Shao1( ), Yin Yang3, Kun Zhou1
State Key Lab of CAD&CG, Zhejiang University, Hangzhou 310058, China.
ShanghaiTech University, Shanghai 200031, China.
The University of New Mexico, Albuquerque, NM 87131, USA.

Abstract

Inferring the functionality of an object from a single RGBD image is difficult for two reasons: lack of semantic information about the object, and missing data due to occlusion. In this paper, we present an interactive framework to recover a 3D functional prototype from a single RGBD image. Instead of precisely reconstructing the object geometry for the prototype, we mainly focus on recovering the object’s functionality along with its geometry. Our system allows users to scribble on the image to create initial rough proxies for the parts. After user annotation of high-level relations between parts, our system automatically jointly optimizes detailed joint parameters (axis and position) and part geometry parameters (size, orientation, and position). Such prototype recovery enables a better understanding of the underlying image geometry and allows for further physically plausible manipulation. We demonstrate our framework on various indoor objects with simple or hybrid functions.

Keywords: prototype, functionality, cuboid proxy, part relations, shape analysis

References(26)

[1]
Han, Y.; Lee, J.-Y.; Kweon, I. S. High quality shape from a single RGB-D image under uncalibrated natural illumination. In: Proceedings of IEEE International Conference on Computer Vision, 1617-1624, 2013.
DOI
[2]
Izadi, S.; Kim, D.; Hilliges, O.; Molyneaux, D.; Newcombe, R.; Kohli, P.; Shotton, J.; Hodges, S.; Freeman, D.; Davison, A.; Fitzgibbon, A. KinectFusion: Real-time 3D reconstruction and interaction using a moving depth camera. In: Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology, 559-568, 2011.
DOI
[3]
Shao, T.; Monszpart, A.; Zheng, Y.; Koo, B.; Xu, W.; Zhou, K.; Mitra, N. J. Imagining the unseen: Stability-based cuboid arrangements for scene understanding. ACM Transactions on Graphics Vol. 33, No. 6, ArticleNo. 209, 2014.
DOI Google Scholar
[4]
Shen, C.-H.; Fu, H.; Chen, K.; Hu, S.-M. Structure recovery by part assembly. ACM Transactions on Graphics Vol. 31, No. 6, Article No. 180, 2012.
DOI Google Scholar
[5]
Zheng, Y.; Chen, X.; Cheng, M.-M.; Zhou, K.; Hu, S.-M.; Mitra, N. J. Interactive images: Cuboid proxies for smart image manipulation. ACM Transactions on Graphics Vol. 31, No. 4, Article No. 99, 2012.
DOI Google Scholar
[6]
Sullivan, L. H. The tall office building artistically considered. Lippincott’s Magazine 57, 1896.
Google Scholar
[7]
Koo, B.; Li, W.; Yao, J.; Agrawala, M.; Mitra, N. J. Creating works-like prototypes of mechanical objects. ACM Transactions on Graphics Vol. 33, No. 6, Article No. 217, 2014.
DOI Google Scholar
[8]
Li, Y.; Wu, X.; Chrysanthou, Y.; Sharf, A.; Cohen-Or, D.; Mitra, N. J. GlobFit: Consistently fitting primitives by discovering global relations. ACM Transactions on Graphics Vol. 30, No. 4, Article No. 52, 2011.
DOI Google Scholar
[9]
Lafarge, F.; Alliez, P. Surface reconstruction through point set structuring. Computer Graphics Forum Vol. 32, No. 2pt2, 225-234, 2013.
DOI Google Scholar
[10]
Arikan, M.; Schwärzler, M.; Flöry, S.; Wimmer, M.; Maierhofer, S. O-snap: Optimization-based snapping for modeling architecture. ACM Transactions on Graphics Vol. 32, No. 1, Article No. 6, 2013.
DOI Google Scholar
[11]
Gupta, A.; Efros, A. A.; Hebert, M. Blocks world revisited: Image understanding using qualitative geometry and mechanics. In: Lecture Notes in Computer Science, Vol. 6314. Daniilidis, K.; Maragos, P.; Paragios, N. Eds. Springer Berlin Heidelberg, 482-496, 2010.
[12]
Gupta, A.; Hebert, M.; Kanade, T.; Blei, D. M. Estimating spatial layout of rooms using volumetric reasoning about objects and surfaces. In: Advances in Neural Information Processing Systems 23. Lafferty, J.; Williams, C.; Shawe-Taylor, J.; Zemel, R.; Culotta, A. Eds. Curran Associates, Inc., 1288-1296, 2010.
[13]
Del Pero, L.; Bowdish, J.; Fried, D.; Kermgard, B.; Hartley, E.; Barnard, K. Bayesian geometric modeling of indoor scenes. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2719-2726, 2012.
DOI
[14]
Jia, Z.; Gallagher, A.; Saxena, A.; Chen, T. 3D-based reasoning with blocks, support, and stability. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 1-8, 2013.
DOI
[15]
Jiang, H.; Xiao, J. A linear approach to matching cuboids in RGBD images. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2171-2178, 2013.
DOI
[16]
Zheng, B.; Zhao, Y.; Yu, J. C.; Ikeuchi, K.; Zhu, S.-C. Beyond point clouds: Scene understanding by reasoning geometry and physics. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 3127-3134, 2013.
DOI
[17]
Umetani, N.; Igarashi, T.; Mitra, N. J. Guided exploration of physically valid shapes for furniture design. ACM Transactions on Graphics Vol. 31, No. 4, Article No. 86, 2012.
DOI Google Scholar
[18]
Shao, T.; Li, W.; Zhou, K.; Xu, W.; Guo, B.; Mitra, N. J. Interpreting concept sketches. ACM Transactions on Graphics Vol. 32, No. 4, Article No. 56, 2013.
DOI Google Scholar
[19]
Bokeloh, M.; Wand, M.; Seidel, H.-P.; Koltun, V. An algebraic model for parameterized shape editing. ACM Transactions on Graphics Vol. 31, No. 4, Article No. 78, 2012.
DOI Google Scholar
[20]
Gal, R.; Sorkine, O.; Mitra, N. J.; Cohen-Or, D. iWIRES: An analyze-and-edit approach to shape manipulation. ACM Transactions on Graphics Vol. 28, No. 3, Article No. 33, 2009.
DOI Google Scholar
[21]
Xu, W.; Wang, J.; Yin, K.; Zhou, K.; van de Panne, M.; Chen, F.; Guo, B. Joint-aware manipulation of deformable models. ACM Transactions on Graphic Vol. 28, No. 3, Article No. 35, 2009.
DOI Google Scholar
[22]
Daniel, M.; Lucas, M. Towards declarative geometric modelling in mechanics. In: Integrated Design and Manufacturing in Mechanical Engineering. Chedmail, P.; Bocquet, J.-C.; Dornfeld, D. Eds. Springer Netherlands, 427-436, 1997.
DOI
[23]
Yvars, P.-A. Using constraint satisfaction for designing mechanical systems. International Journal on Interactive Design and Manufacturing Vol. 2, No. 3, 161-167, 2008.
DOI Google Scholar
[24]
Zhang, Q.; Ye, M.; Yang, R.; Matsushita, Y.; Wilburn, B.; Yu, H. Edge-preserving photometric stereo via depth fusion. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2472-2479, 2012.
[25]
Rother, C.; Kolmogorov, V.; Blake, A. “GrabCut": Interactive foreground extraction using iterated graph cuts. ACM Transactions on Graphics Vol. 23, No. 3, 309-314, 2004.
DOI Google Scholar
[26]
Schnabel, R.; Wahl, R.; Klein, R. Efficient RANSAC for point-cloud shape detection. Computer Graphics Forum Vol. 26, No. 2, 214-226, 2007.
DOI Google Scholar
Video
41095_2016_32_MOESM1_ESM.mp4
Publication history
Copyright
Rights and permissions

Publication history

Revised: 01 December 2015
Accepted: 09 December 2015
Published: 29 January 2016
Issue date: March 2016

Copyright

© The Author(s) 2016

Rights and permissions

This article is published with open access at Springerlink.com

The articles published in this journal are distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Other papers from this open access journal are available free of charge from http://www.springer.com/journal/41095. To submit a manuscript, please go to https://www.editorialmanager.com/cvmj.

Return