Journal Home > Volume 4 , Issue 1
Computational Visual Media 2018, 4(1): 83-102 https://doi.org/10.1007/s41095-017-0101-9
Research Article | Open Access | Issue | Published: 14 February 2018

Photometric stereo for strong specular highlights

Show Author's Information Maryam Khanian1( ),  Ali Sharifi Boroujerdi1,  Michael Breuß1
Chair of Applied Mathematics, Brandenburg University of Technology, Cottbus 03046, Germany.
Keywords:
photometric stereo (PS), complete Blinn–Phong model, perspective projection, diffuse reflection, specular reflection
Cite this article:
Khanian M, Boroujerdi AS, Breuß M. Photometric stereo for strong specular highlights. Computational Visual Media, 2018, 4(1): 83-102. https://doi.org/10.1007/s41095-017-0101-9
Download citation
EndNote(RIS)
BibTeX

582

Views

17

Downloads

Citations

12

Crossref

N/A

WoS

12

Scopus

2

CSCD

Abstract Full text About this article

Photometric stereo is a fundamental technique in computer vision known to produce 3D shape with high accuracy. It uses several input images of a static scene taken from one and the same camera position but under varying illumination. The vast majority of studies in this 3D reconstruction method assume orthographic projection for the camera model. In addition, they mainly use the Lambertian reflectance model as the way that light scatters at surfaces. Thus, providing reliable photometric stereo results from real world objects still remains a challenging task. We address 3D reconstruction by use of a more realistic set of assumptions, combining for the first time the complete Blinn–Phong reflectance model and perspective projection. Furthermore, we compare two different methods of incorporating the perspective projection into our model. Experiments are performed on both synthetic and real world images; the latter do not benefit from laboratory conditions. The results show the high potential of our method even for complex real world applications such as medical endoscopy images which may include many specular highlights.


menu
Abstract
Full text
Outline
About this article

Photometric stereo for strong specular highlights

Show Author's information Maryam Khanian1( ), Ali Sharifi Boroujerdi1, Michael Breuß1
Chair of Applied Mathematics, Brandenburg University of Technology, Cottbus 03046, Germany.

Abstract

Photometric stereo is a fundamental technique in computer vision known to produce 3D shape with high accuracy. It uses several input images of a static scene taken from one and the same camera position but under varying illumination. The vast majority of studies in this 3D reconstruction method assume orthographic projection for the camera model. In addition, they mainly use the Lambertian reflectance model as the way that light scatters at surfaces. Thus, providing reliable photometric stereo results from real world objects still remains a challenging task. We address 3D reconstruction by use of a more realistic set of assumptions, combining for the first time the complete Blinn–Phong reflectance model and perspective projection. Furthermore, we compare two different methods of incorporating the perspective projection into our model. Experiments are performed on both synthetic and real world images; the latter do not benefit from laboratory conditions. The results show the high potential of our method even for complex real world applications such as medical endoscopy images which may include many specular highlights.

Keywords: photometric stereo (PS), complete Blinn–Phong model, perspective projection, diffuse reflection, specular reflection

References(61)

[1]
Horn, B. K. P. Robot Vision. The MIT Press, 1986.
[2]
Trucco, E.; Verri, A. Introductory Techniques for 3-D Computer Vision. Prentice Hall PTR, 1998.
[3]
Wöhler, C. 3D Computer Vision. Springer-Verlag, 2013.
DOI
[4]
Ihrke, I.; Kutulakos, K. N.; Lensch, H. P. A.; Magnor, M.; Heidrich, W. Transparent and specular object reconstruction. Computer Graphics Forum Vol. 29, No. 8, 2400-2426, 2010.
DOI Google Scholar
[5]
Xiong, Y.; Shafer, S. A. Depth from focusing and defocusing. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 68-73, 1993.
DOI
[6]
Faugeras, O. Three-Dimensional Computer Vision. The MIT Press, 1993.
[7]
Tomasi, C.; Kanade, T. Shape and motion from image streams under orthography: A factorization method. International Journal of Computer Vision Vol. 9, No. 2, 137-154, 1992.
DOI Google Scholar
[8]
Adato, Y.; Vasilyev, Y.; Zickler, T.; Ben-Shahar, O. Shape from specular flow. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 32, No. 11, 2054-2070, 2010.
DOI Google Scholar
[9]
Godard, C.; Hedman, P.; Li, W.; Brostow, G. J. Multi-view reconstruction of highly specular surfaces in uncontrolled environments. In: Proceedings of the International Conference on 3D Vision, 19-27, 2015.
DOI
[10]
Sankaranarayanan, A. C.; Veeraraghavan, A.; Tuzel, O.; Agrawal, A. Specular surface reconstruction from sparse reflection correspondences. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 1245-1252, 2010.
DOI
[11]
Woodham, R. J. Photometric stereo: A reflectance map technique for determining surface orientation from image intensity. In: Proceedings of the SPIE 0155, Image Understanding Systems and Industrial Applications I, 136-143, 1978.
DOI
[12]
Horn, B. K. P.; Woodham, R. J.; Silver, W. M. Determining shape and reflectance using multiple images. MIT Artificial Intelligence Laboratory, Memo 490, 1978.
[13]
Woodham, R. J. Photometric method for determining surface orientation from multiple images. Optical Engineering Vol. 19, No. 1, 134-144, 1980.
DOI Google Scholar
[14]
Lambert, J. H.; DiLaura, D. L. Photometry, or, on the measure and gradations of light, colors, and shade: Translation from the Latin of photometria, sive, de mensura et gradibus luminis, colorum et umbrae. Illuminating Engineering Society of North America, 2001.
[15]
Beckmann, P.; Spizzichino, A. The Scattering of Electromagnetic Waves from Rough Surfaces. Norwood, MA, USA: Artech House, Inc., 1987.
[16]
Brandenberg, W. M.; Neu, J. T. Undirectional reflectance of imperfectly diffuse surfaces. Journal of the Optical Society of America Vol. 56, No. 1, 97-103, 1966.
DOI Google Scholar
[17]
Tagare, H. D.; Defigueiredo, R. J. P. A framework for the construction of general reflectance maps for machine vision. CVGIP: Image Understanding Vol. 57, No. 3, 265-282, 1993.
DOI Google Scholar
[18]
Tankus, A.; Sochen, N.; Yeshurun, Y. Shape-from-shading under perspective projection. International Journal of Computer Vision Vol. 63, No. 1, 21-43, 2005.
DOI Google Scholar
[19]
Mukaigawa, Y.; Ishii, Y.; Shakunaga, T. Analysis of photometric factors based on photometric linearization. Journal of the Optical Society of America A Vol. 24, No. 10, 3326-3334, 2007.
DOI Google Scholar
[20]
Mallick, S. P.; Zickler, T. E.; Kriegman, D. J.; Belhumeur, P. N. Beyond Lambert: Reconstructing specular surfaces using color. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Vol. 2, 619-626, 2005.
[21]
Yu, C.; Seo, Y.; Lee, S. W. Photometric stereo from maximum feasible Lambertian reflections. In: Computer Vision – ECCV 2010. Lecture Notes in Computer Science, Vol, 6314. Daniilidis, K.; Maragos, P.; Paragios, N. Eds. Springer, Berlin, Heidelberg, 115-126, 2010.
DOI
[22]
Miyazaki, D.; Hara, K.; Ikeuchi, K. Median photometric stereo as applied to the segonko tumulus and museum objects. International Journal of Computer Vision Vol. 86, Nos. 2–3, 229-242, 2010.
DOI Google Scholar
[23]
Tang, K.-L.; Tang, C.-K.; Wong, T.-T. Dense photometric stereo using tensorial belife propagation. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Vol. 1, 132-139, 2005.
[24]
Wu, L.; Ganesh, A.; Shi, B.; Matsushita, Y.; Wang, Y.; Ma, Y. Robust photometris stereo via low-rank matrix completion and recovery. In: Computer Vision – ACCV 2010. Lecture Notes in Computer Science, Vol. 6494. Kimmel, R.; Klette, R.; Sugimoto, A. Eds. Springer, Berlin, Heidelberg, 703-717, 2010.
[25]
Smith, W.; Fang, F. Height from photometric ratio with model-based light source selection. Computer Vision and Image Understanding Vol. 145, 128-138, 2016.
DOI Google Scholar
[26]
Hertzmann, A.; Seitz, S. M. Shape and materials by example: A photometric stereo approach. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Vol. 1, I-533–I-540, 2003.
[27]
Goldman, D. B.; Curless, B.; Hertzmann, A.; Seitz, S. M. Shape and spatially-varying BRDFs from photometric stereo. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 32, No. 6, 1060-1071, 2010.
DOI Google Scholar
[28]
Oxholm, G.; Nishino, K. Multiview shape and reflectance from natural illumination. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2163-2170, 2014.
DOI
[29]
Galo, M.; Tozzi, C. L. Surface reconstruction using multiple light sources and perspective projection. In: Proceedings of the 3rd IEEE International Conference on Image Processing, Vol. 2, 309-312, 1996.
[30]
Tankus, A.; Kiryati, N. Photometric stereo under perspective projection. In: Proceedings of the 10th IEEE International Conference on Computer Vision, Vol. 1, 611-616, 2005.
DOI
[31]
Mecca, R.; Tankus, A; Bruckstein, A. M. Two-image perspective photometric stereo using shape-from-shading. In: Computer Vision – ACCV 2012. Lecture Notes in Computer Science, Vol. 7727. Lee, K. M.; Matsushita, Y.; Rehg, J. M.; Hu, Z. Eds. Springer, Berlin, Heidelberg, 110-121, 2013.
DOI
[32]
Vogel, O.; Valgaerts, L.; Breuß, M.; Weickert, J. Making shape from shading work for real-world images. In: Pattern Recognition. Lecture Notes in Computer Science, Vol. 5748. Denzler, J.; Notni, G.; Süße, H. Eds. Springer, Berlin, Heidelberg, 191-200, 2009.
DOI
[33]
Cho, S.-Y.; Chow, T. W. S. Shape recovery from shading by a new neural-based reflectance model. IEEE Transactions on Neural Networks Vol. 10, No. 6, 1536-1541, 1999.
DOI Google Scholar
[34]
Blinn, J. F. Models of light reflection for computer synthesized pictures. In: Proceedings of the 4th Annual Conference on Computer Graphics and Interactive Techniques, 192-198, 1977.
DOI
[35]
Phong, B. T. Illumination for computer generated pictures. Communications of ACM Vol. 18, No. 6, 311-317, 1975.
DOI Google Scholar
[36]
Hartley, R.; Zisserman, A. Multiple View Geometry in Computer Vision. Cambridge University Press, 2003.
DOI
[37]
Mecca, R.; Rodolà, E.; Cremers, D. Realistic photometric stereo using partial differential irradiance equation ratios. Computers & Graphics Vol. 51, 8-16, 2015.
DOI Google Scholar
[38]
Mecca, R.; Quéau, Y. Unifying diffuse and specular reflections for the photometric stereo problem. In: Proceedings of the IEEE Winter Conference on Applications of Computer Vision, 1-9, 2016.
DOI
[39]
Tozza, S.; Mecca, R.; Duocastella, M.; Del Bue, A. Direct differential photometric stereo shape recovery of diffuse and specular surfaces. Journal of Mathematical Imaging and Vision Vol. 56, No. 1, 57-76, 2016.
DOI Google Scholar
[40]
Kim, H.; Jin, H.; Hadap, S.; Kweon, K. Specular reflection separation using dark channel prior. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1460-1467, 2013.
DOI
[41]
Mallick, S. P.; Zickler, T. E.; Kriegman, D. J.; Belhumeur, P. N. Beyond Lambert: Reconstructing specular surfaces using color. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Vol. 2, 619-626, 2005.
[42]
Tan, R. T.; Ikeuchi, K. Separating reflection components of textured surfaces using a single image. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol.27, No. 2, 178-193, 2005.
DOI Google Scholar
[43]
Khanian, M.; Sharifi Boroujerdi, A.; Breuß, M. Perspective photometric stereo beyond Lambert. In: Proceedings of Vol. 9534, the 12th International Conference on Quality Control by Artificial Vision, 95341F, 2015.
DOI
[44]
Papadhimitri, T.; Favaro, P. A new perspective on uncalibrated photometric stereo. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1474-1481, 2013.
DOI
[45]
Quéau, Y.; Durou, J.-D. Edge-preserving integration of a normal field: Weighted least-squares, TV and L1 approaches. In: Scale Space and Variational Methods in Computer Vision. Lecture Notes in Computer Science, Vol. 9087. Aujol, J. F.; Nikolova, M.; Papadakis, N. Eds. Springer, Cham, 576-588, 2015.
[46]
Camilli, F.; Tozza, S. A unified approach to the well-posedness of some non-Lambertian models in shape-from-shading. SIAM Journal on Imaging Sciences Vol. 10, No. 1, 26-46, 2017.
DOI Google Scholar
[47]
Levenberg, K. A method for the solution of certain non-linear problems in least squares. Quarterly of Applied Mathematics Vol. 2, No. 2, 164-168, 1944.
DOI Google Scholar
[48]
Marquardt, D. An algorithm for least squares estimation on nonlinear parameters. Journal of the Society of Industrial and Applied Mathematics Vol. 11, No. 2, 431-441, 1963.
DOI Google Scholar
[49]
Bähr, M.; Breuß, M.; Quéau, Y.; Boroujerdi, A. S.; Durou, J.-D. Fast and accurate surface normal integration on non-rectangular domains. Computational Visual Media Vol. 3, No. 2, 107-129, 2017.
DOI Google Scholar
[50]
The Stanford 3D scanning repository. Available at http://graphics.stanford.edu/data/3Dscanrep/.
[51]
Sumner, R. W.; Popović, J. Deformation transfer for triangle meshes. ACM Transactions on Graphics Vol. 23, No. 3, 399-405, 2004.
DOI Google Scholar
[52]
Norman, J. F.; Todd, J. T.; Norman, H. F.; Clayton, A. M.; McBride, T. R. Visual discrimination of local surface structure: Slant, tilt, and curvedness. Vision Research Vol. 46, Nos. 6–7, 1057-1069, 2006.
DOI Google Scholar
[53]
Rosenberg, A.; Cowan, N. J.; Angelaki, D. E. The visual representation of 3D object orientation in parietal cortex. Journal of Neuroscience Vol. 33, No. 49, 19352-19361, 2013.
DOI Google Scholar
[54]
Sugihara, H.; Murakami, I.; Shenoy, K. V.; Andersen, R. A.; Komatsu, H. Response of MSTD neurons to simulated 3D orientation of rotating planes. Journal of Neurophysiology Vol. 87, No. 1, 273-285, 2002.
DOI Google Scholar
[55]
Saunders, J. A.; Knill, D. C. Perception of 3D surface orientation from skew symmetry. Vision Research Vol. 41, No. 24, 3163-3183, 2001.
DOI Google Scholar
[56]
Stevens, K. A. Surface tilt (the direction of slant): A neglected psychophysical variable. Perception & Psychophysics Vol. 33, No. 3, 241-250, 1983.
DOI Google Scholar
[57]
Braunstein, M. L.; Payne, J. W. Perspective and form ratio as determinants of relative slant judgments. Journal of Experimental Psychology Vol. 81, No. 3, 584-590, 1969.
DOI Google Scholar
[58]
Tibau, S.; Willems, B.; Van Den Bergh, E.; Wagemans, J. The role of the centre of projection in the estimation of slant from texture of planar surfaces. Perception Vol. 30, No. 2, 185-193, 2001.
DOI Google Scholar
[59]
Tankus, A.; Sochen, N.; Yeshurun, Y. Reconstruction of medical images by perspective shape-from-shading. In: Proceedings of the 17th International Conference on Pattern Recognition, Vol. 3, 778-781, 2004.
DOI
[60]
Tatemasu, K.; Iwahori, Y.; Nakamura, T.; Fukui, S.; Woodham, R. J.; Kasugai, K. Shape from endoscope image based on photometric and geometric constraints. Procedia Computer Science Vol. 22, 1285-1293, 2013.
DOI Google Scholar
[61]
Pharr, M.; Jakob, W.; Humphreys, G. Physically Based Rendering: From Theory to Implementation. Morgan Kaufmann Publishers Inc., 2010.
Publication history
Copyright
Acknowledgements
Rights and permissions

Publication history

Revised: 04 September 2017
Accepted: 01 December 2017
Published: 14 February 2018
Issue date: March 2018

Copyright

© The Author(s) 2017

Acknowledgements

This work was supported by the Deutsche Forschungsgemeinschaft under grant number BR2245/4–1. The authors would like to thank the anonymous reviewers for helpful comments to improve the quality of the paper.

Rights and permissions

This article is published with open access at Springerlink.com

The articles published in this journal are distributed under the terms of the Creative Commons Attribution 4.0 International License (http:// creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Other papers from this open access journal are available free of charge from http://www.springer.com/journal/41095. To submit a manuscript, please go to https://www. editorialmanager.com/cvmj.

Return