Journal Home > Volume 4 , Issue 3
Computational Visual Media 2018, 4(3): 267-276 https://doi.org/10.1007/s41095-018-0113-0
Research Article | Open Access | Issue | Published: 04 April 2018

Instantaneous foveated preview for progressive Monte Carlo rendering

Show Author's Information Matias K. Koskela1( ),  Kalle V. Immonen1,  Timo T. Viitanen1,  Pekka O. Jääskeläinen1,  Joonas I. Multanen1,  Jarmo H. Takala1
Tampere University of Technology, Tampere, 33720, Finland.
Keywords:
Monte Carlo rendering, foveated rendering, progressive rendering, preview, 360∘ content
Cite this article:
Koskela MK, Immonen KV, Viitanen TT, et al. Instantaneous foveated preview for progressive Monte Carlo rendering. Computational Visual Media, 2018, 4(3): 267-276. https://doi.org/10.1007/s41095-018-0113-0
Download citation
EndNote(RIS)
BibTeX

635

Views

20

Downloads

Citations

6

Crossref

N/A

WoS

6

Scopus

1

CSCD

Abstract Full text Electronic supplementary material About this article

Progressive rendering, for example Monte Carlo rendering of 360∘ content for virtual reality headsets, is a time-consuming task. If the 3D artist notices an error while previewing the rendering, they must return to editing mode, make the required changes, and restart rendering. We propose the use of eye-tracking-based optimization to significantly speed up previewing of the artist’s points of interest. The speed of the preview is further improved by sampling with a distribution that closely follows the experimentally measured visual acuity of the human eye, unlike the piecewise linear models used in previous work. In a comprehensive user study, the perceived convergence of our proposed method was 10 times faster than that of a conventional preview, and often appeared to be instantaneous. In addition, the participants rated the method to have only marginally more artifacts in areas where it had to start rendering from scratch, compared to conventional rendering methods that had already generated image content in those areas.


menu
Abstract
Full text
Outline
Electronic supplementary material
About this article

Instantaneous foveated preview for progressive Monte Carlo rendering

Show Author's information Matias K. Koskela1( ), Kalle V. Immonen1, Timo T. Viitanen1, Pekka O. Jääskeläinen1, Joonas I. Multanen1, Jarmo H. Takala1
Tampere University of Technology, Tampere, 33720, Finland.

Abstract

Progressive rendering, for example Monte Carlo rendering of 360∘ content for virtual reality headsets, is a time-consuming task. If the 3D artist notices an error while previewing the rendering, they must return to editing mode, make the required changes, and restart rendering. We propose the use of eye-tracking-based optimization to significantly speed up previewing of the artist’s points of interest. The speed of the preview is further improved by sampling with a distribution that closely follows the experimentally measured visual acuity of the human eye, unlike the piecewise linear models used in previous work. In a comprehensive user study, the perceived convergence of our proposed method was 10 times faster than that of a conventional preview, and often appeared to be instantaneous. In addition, the participants rated the method to have only marginally more artifacts in areas where it had to start rendering from scratch, compared to conventional rendering methods that had already generated image content in those areas.

Keywords: Monte Carlo rendering, foveated rendering, progressive rendering, preview, 360∘ content

References(26)

[1]
Pharr, M.; Jakob, W.; Humphreys, G. Physically Based Rendering: From Theory to Implementation, 2nd edn. Morgan Kaufmann, 2010.
[2]
Strasburger, H.; Rentschler, I.; Jüttner, M. Peripheral vision and pattern recognition: A review. Journal of Vision Vol. 11, No. 5, 13, 2011.
DOI Google Scholar
[3]
Koskela, M.; Viitanen, T.; Jääskeläinen, P.; Takala, J. Foveated path tracing. In: Advances in Visual Computing. Lecture Notes in Computer Science, Vol. 10072. Bebis, G.; Boyle, R.; Parvin, B. et al. Eds. Springer Cham, 723–732, 2016.
[4]
Koskela, M.; Immonen, K.; Viitanen, T.; Jääskeläinen, P.; Multanen, J.; Takala, J. Foveated instant preview for progressive rendering. In: Proceedings of the SIGGRAPH Asia 2017 Technical Briefs, Article No. 10, 2017.
DOI
[5]
Weier, M.; Stengel, M.; Roth, T.; Didyk, P.; Eisemann, E.; Eisemann, M.; Grogorick, S.; Hinkenjann, A.; Kruijff, E.; Magnor, M.; Myszkowski, K.; Slusallek, P. Perception-driven accelerated rendering. Computer Graphics Forum Vol. 36, No. 2, 611–643, 2017.
DOI Google Scholar
[6]
Shibata, T. Head mounted display. Displays Vol. 23, Nos. 1–2, 57–64, 2002.
DOI Google Scholar
[7]
Lee, E. C.; Park, K. R. A robust eye gaze tracking method based on a virtual eyeball model. Machine Vision and Applications Vol. 20, No. 5, 319–337, 2009.
DOI Google Scholar
[8]
Guenter, B.; Finch, M.; Drucker, S.; Tan, D.; Snyder, J. Foveated 3D graphics. ACM Transactions on Graphics Vol. 31, No. 6, Article No. 164, 2012.
DOI Google Scholar
[9]
Vaidyanathan, K.; Salvi, M.; Toth, R.; Foley, T.; Akenine-Möller, T.; Nilsson, J.; Munkberg, J.; Hasselgren, J.; Sugihara, M.; Clarberg, P.; Janczak, T.; Lefohn, A. Coarse pixel shading. In: Proceedings of High Performance Graphics, 9–18, 2014.
[10]
Stengel, M.; Grogorick, S.; Eisemann, M.; Magnor, M. Adaptive image-space sampling for gaze-contingent real-time rendering. Computer Graphics Forum Vol. 35, No. 4, 129–139, 2016.
DOI Google Scholar
[11]
Weier, M.; Roth, T.; Kruijff, E.; Hinkenjann, A.; Pérard-Gayot, A.; Slusallek, P.; Li, Y. Foveated real-time ray tracing for head-mounted displays. Computer Graphics Forum Vol. 35, No. 7, 289–298, 2016.
DOI Google Scholar
[12]
Murphy, H. A.; Duchowski, A. T.; Tyrrell, R. A. Hybrid image/model-based gaze-contingent rendering. ACM Transactions on Applied Perception Vol. 5, No. 4, Article No. 22, 2009.
DOI Google Scholar
[13]
Reddy, M. Perceptually optimized 3D graphics. IEEE Computer Graphics and Applications Vol. 21, No. 5, 68–75, 2001.
DOI Google Scholar
[14]
Pohl, D.; Zhang, X.; Bulling, A. Combining eye tracking with optimizations for lens astigmatism in modern wide-angle HMDs. In: Proceedings of the IEEE Virtual Reality, 269–270, 2016.
DOI
[15]
Roth, T.; Weier, M.; Maiero, J.; Hinkenjann, A.; Li, Y. Guided high-quality rendering. In: Advances in Visual Computing. Lecture Notes in Computer Science, Vol. 9475. Bebis, G.; Boyle, R.; Parvin, B. et al. Eds. Springer Cham, 115–125, 2015.
DOI
[16]
Pixar. Renderman 20 documentation: Rendering efficiently. 2017. Available at https://renderman.pixar.com/resources/RenderMan_20/tutorialRenderingEfficiently.html.
[17]
The community of LuxRender. LuxRender documentation: Refine brush. 2013. Available at http://www.luxrender.net/wiki/Refine_Brush.
[18]
Duchowski, A. T.; Bate, D.; Stringfellow, P.; Thakur, K.; Melloy, B. J.; Gramopadhye, A. K. On spatiochromatic visual sensitivity and peripheral color LOD management. ACM Transactions on Applied Perception Vol. 6, No. 2, Article No. 9, 2009.
DOI Google Scholar
[19]
Viitanen, T.; Koskela, M.; Immonen, K.; Mäkitalo, M.; Jääskeläinen, P.; Takala, J. Sparse sampling for real-time ray tracing. In: Proceedings of the 13th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications, Vol. 1, 295–302, 2018.
DOI
[20]
Devroye, L. Non-Uniform Random Variate Generation. Springer-Verlag, 1986.
DOI
[21]
Weisstein, E. Lambert W-function. Available at http://mathworld.wolfram.com/LambertW-Function.html.
[22]
MacQuarrie, A.; Steed, A. Cinematic virtual reality: Evaluating the effect of display type on the viewing experience for panoramic video. In: Proceedings of the IEEE Virtual Reality, 45–54, 2017.
DOI
[23]
Stark, R.; Israel, J. H.; Wöhler, T. Towards hybrid modelling environments—Merging desktop-CAD and virtual reality-technologies. CIRP Annals Vol. 59, No. 1, 179–182, 2010.
DOI Google Scholar
[24]
AMD. Radeon Rays SDK. Available at https://github.com/GPUOpen-LibrariesAndSDKs/RadeonRays_SDK.
[25]
Rodrigo. What is the latency of FOVE eye tracking? Available at https://support.getfove.com/hc/en-us/articles/115000733714-What-is-the-Latency-of-FOVE-Eye-Tracking-.
[26]
FOVE Inc. Tech specs. Available at https://www.getfov.com/.
Video
CVM_2018_3_267-276_ESM.mp4
Publication history
Copyright
Acknowledgements
Rights and permissions

Publication history

Revised: 23 December 2017
Accepted: 17 February 2018
Published: 04 April 2018
Issue date: September 2018

Copyright

© The Author(s) 2018

Acknowledgements

The authors would like to thank the creators of the 3D models used in the user study: Christophe Seux for Classroom, Anat Grynberg and Greg Ward for Conference, Marko Dabrovic for Sibenik (License: CC BY-NC), and Frank Meinl for Crytek Sponza (License: CC BY 3.0). In addition, the authors would like to thank Heli Väätäjä, Chelsea Kelling, and Otto Kauhanen for helpful discussions.

The work was financially supported by the TUT Graduate School, Nokia Foundation, Emil Aaltonen Foundation, Finnish Foundation for Technology Promotion, Academy of Finland (funding decision 297548), Finnish Funding Agency for Technology and Innovation (funding decision 40142/14, FiDiPro-StreamPro) and ARTEMIS joint undertaking under grant agreement no 621439 (ALMARVI).

Rights and permissions

This article is published with open access at Springerlink.com

The articles published in this journal are distributed under the terms of the Creative Commons Attribution 4.0 International License (http:// creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Other papers from this open access journal are available free of charge from http://www.springer.com/journal/41095. To submit a manuscript, please go to https://www. editorialmanager.com/cvmj.

Return