Semantic Segmentation of 3D Point Cloud to Virtually Manipulate Real Living Space

Yuki Ishikawa, Ryo Hachiuma, Naoto Ienaga, Wakaba Kuno, Yuta Sugiura, Hideo Saito

Research output: Chapter in Book/Report/Conference proceedingConference contribution

2 Citations (Scopus)

Abstract

This paper presents a method for the virtual manipulation of real living space using semantic segmentation of a 3D point cloud captured in the real world. We applied PointNet to segment each piece of furniture from the point cloud of a real indoor environment captured by moving a RGB-D camera. For semantic segmentation, we focused on local geometric information not used in PointNet, and we proposed a method to refine the class probability of labels attached to each point in PointNet's output. The effectiveness of our method was experimentally confirmed. We then created 3D models of real-world furniture using a point cloud with corrected labels, and we virtually manipulated real living space using Dollhouse VR, a layout system.

Original languageEnglish
Title of host publicationProceedings of the 2019 12th Asia Pacific Workshop on Mixed and Augmented Reality, APMAR 2019
EditorsDongdong Weng, Liwei Chan, Youngho Lee, Xiaohui Liang, Nobuchika Sakata
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781728115719
DOIs
Publication statusPublished - 2019 May 7
Event12th Asia Pacific Workshop on Mixed and Augmented Reality, APMAR 2019 - Ikoma, Nara, Japan
Duration: 2019 Mar 282019 Mar 29

Publication series

NameProceedings of the 2019 12th Asia Pacific Workshop on Mixed and Augmented Reality, APMAR 2019

Conference

Conference12th Asia Pacific Workshop on Mixed and Augmented Reality, APMAR 2019
CountryJapan
CityIkoma, Nara
Period19/3/2819/3/29

Keywords

  • furniture arrangement
  • semantic segmentation
  • virtual reality

ASJC Scopus subject areas

  • Education
  • Sociology and Political Science
  • Communication
  • Marketing
  • Strategy and Management
  • Computer Networks and Communications
  • Information Systems and Management
  • Social Psychology

Fingerprint Dive into the research topics of 'Semantic Segmentation of 3D Point Cloud to Virtually Manipulate Real Living Space'. Together they form a unique fingerprint.

  • Cite this

    Ishikawa, Y., Hachiuma, R., Ienaga, N., Kuno, W., Sugiura, Y., & Saito, H. (2019). Semantic Segmentation of 3D Point Cloud to Virtually Manipulate Real Living Space. In D. Weng, L. Chan, Y. Lee, X. Liang, & N. Sakata (Eds.), Proceedings of the 2019 12th Asia Pacific Workshop on Mixed and Augmented Reality, APMAR 2019 [8709156] (Proceedings of the 2019 12th Asia Pacific Workshop on Mixed and Augmented Reality, APMAR 2019). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/APMAR.2019.8709156