Mobile Photometric Stereo with Keypoint-Based SLAM for Dense 3D Reconstruction

Remy Maxence, Hideaki Uchiyama, Hiroshi Kawasaki, Diego Thomas, Vincent Nozick, Hideo Saito

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

The standard photometric stereo is a technique to densely reconstruct objects' surfaces using light variation under the assumption of a static camera with a moving light source. In this work, we use photometric stereo to reconstruct dense 3D scenes while moving the camera and the light altogether. In such non-static case, camera poses as well as correspondences between pixels of each frame to apply photometric stereo are required. ORB-SLAM is a technique that can be used to estimate camera poses. To retrieve correspondences, our idea is to start from a sparse 3D mesh obtained with ORB SLAM and then densify the mesh by a plane sweep method using a multi-view photometric consistency. By combining ORB-SLAM and photometric stereo, it is possible to reconstruct dense 3D scenes with a off-the-shelf smartphone and its embedded torchlight. Note that SLAM systems usually struggle with textureless object, which is effectively compensated by the photometric stereo in our method. Experiments are conducted to show that our proposed method gives better results than SLAM alone or COLMAP, especially for partially textureless surfaces.

Original languageEnglish
Title of host publicationProceedings - 2019 International Conference on 3D Vision, 3DV 2019
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages574-582
Number of pages9
ISBN (Electronic)9781728131313
DOIs
Publication statusPublished - 2019 Sep
Event7th International Conference on 3D Vision, 3DV 2019 - Quebec, Canada
Duration: 2019 Sep 152019 Sep 18

Publication series

NameProceedings - 2019 International Conference on 3D Vision, 3DV 2019

Conference

Conference7th International Conference on 3D Vision, 3DV 2019
CountryCanada
CityQuebec
Period19/9/1519/9/18

Keywords

  • 3D reconstruction
  • ORB SLAM
  • SLAM
  • dynamic
  • photometric
  • photometric stereo
  • plane sweep
  • smartphone

ASJC Scopus subject areas

  • Computer Vision and Pattern Recognition
  • Media Technology
  • Modelling and Simulation

Fingerprint Dive into the research topics of 'Mobile Photometric Stereo with Keypoint-Based SLAM for Dense 3D Reconstruction'. Together they form a unique fingerprint.

  • Cite this

    Maxence, R., Uchiyama, H., Kawasaki, H., Thomas, D., Nozick, V., & Saito, H. (2019). Mobile Photometric Stereo with Keypoint-Based SLAM for Dense 3D Reconstruction. In Proceedings - 2019 International Conference on 3D Vision, 3DV 2019 (pp. 574-582). [8885679] (Proceedings - 2019 International Conference on 3D Vision, 3DV 2019). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/3DV.2019.00069