Efficient object-oriented semantic mapping with object detector

Yoshikatsu Nakajima, Hideo Saito

Research output: Contribution to journalArticle

1 Citation (Scopus)

Abstract

Incrementally, building a 3D map in which object instances are semantically annotated has a wide range of applications, including scene understanding, human-robot interactions, and simultaneous localization and mapping extensions. Although researchers are developing efficient and accurate systems, these methods still face a critical issue: Real-time processing, because the task requires a series of heavy processing components, e.g., camera pose estimation, 3D map reconstruction, and especially recognition. In this paper, we propose a novel object-oriented semantic mapping approach aiming at overcoming such issues by introducing highly accurate object-oriented semantic scene reconstruction in real time. For high efficiency, the proposed method employs a fast and scalable object detection algorithm for exploiting semantic information from the incoming frames. These outputs are integrated into geometric regions of the 3D map, which are carried by the geometric-based incremental segmentation method. The strategy of assigning class probabilities to each segmented region, not each element (e.g., surfels and voxels), notably reduces the computational cost, as well as the memory footprint. In addition to efficiency, by geometrically segmenting the 3D map first, clear boundaries between objects appear. We complementarily improve the geometric-based segmentation results beyond the geometric only to the semantic-aware representation. We validate the proposed method's accuracy and computational efficiency through experiments in a common office scene.

Original languageEnglish
Article number8579143
Pages (from-to)3206-3213
Number of pages8
JournalIEEE Access
Volume7
DOIs
Publication statusPublished - 2019 Jan 1

Fingerprint

Semantics
Detectors
Human robot interaction
Processing
Computational efficiency
Cameras
Data storage equipment
Costs
Experiments

Keywords

  • object detection
  • segmentation
  • Semantic mapping
  • SLAM

ASJC Scopus subject areas

  • Computer Science(all)
  • Materials Science(all)
  • Engineering(all)

Cite this

Efficient object-oriented semantic mapping with object detector. / Nakajima, Yoshikatsu; Saito, Hideo.

In: IEEE Access, Vol. 7, 8579143, 01.01.2019, p. 3206-3213.

Research output: Contribution to journalArticle

Nakajima, Yoshikatsu ; Saito, Hideo. / Efficient object-oriented semantic mapping with object detector. In: IEEE Access. 2019 ; Vol. 7. pp. 3206-3213.
@article{a8036e7314d6410bbcf4f08d4adb39ee,
title = "Efficient object-oriented semantic mapping with object detector",
abstract = "Incrementally, building a 3D map in which object instances are semantically annotated has a wide range of applications, including scene understanding, human-robot interactions, and simultaneous localization and mapping extensions. Although researchers are developing efficient and accurate systems, these methods still face a critical issue: Real-time processing, because the task requires a series of heavy processing components, e.g., camera pose estimation, 3D map reconstruction, and especially recognition. In this paper, we propose a novel object-oriented semantic mapping approach aiming at overcoming such issues by introducing highly accurate object-oriented semantic scene reconstruction in real time. For high efficiency, the proposed method employs a fast and scalable object detection algorithm for exploiting semantic information from the incoming frames. These outputs are integrated into geometric regions of the 3D map, which are carried by the geometric-based incremental segmentation method. The strategy of assigning class probabilities to each segmented region, not each element (e.g., surfels and voxels), notably reduces the computational cost, as well as the memory footprint. In addition to efficiency, by geometrically segmenting the 3D map first, clear boundaries between objects appear. We complementarily improve the geometric-based segmentation results beyond the geometric only to the semantic-aware representation. We validate the proposed method's accuracy and computational efficiency through experiments in a common office scene.",
keywords = "object detection, segmentation, Semantic mapping, SLAM",
author = "Yoshikatsu Nakajima and Hideo Saito",
year = "2019",
month = "1",
day = "1",
doi = "10.1109/ACCESS.2018.2887022",
language = "English",
volume = "7",
pages = "3206--3213",
journal = "IEEE Access",
issn = "2169-3536",
publisher = "Institute of Electrical and Electronics Engineers Inc.",

}

TY - JOUR

T1 - Efficient object-oriented semantic mapping with object detector

AU - Nakajima, Yoshikatsu

AU - Saito, Hideo

PY - 2019/1/1

Y1 - 2019/1/1

N2 - Incrementally, building a 3D map in which object instances are semantically annotated has a wide range of applications, including scene understanding, human-robot interactions, and simultaneous localization and mapping extensions. Although researchers are developing efficient and accurate systems, these methods still face a critical issue: Real-time processing, because the task requires a series of heavy processing components, e.g., camera pose estimation, 3D map reconstruction, and especially recognition. In this paper, we propose a novel object-oriented semantic mapping approach aiming at overcoming such issues by introducing highly accurate object-oriented semantic scene reconstruction in real time. For high efficiency, the proposed method employs a fast and scalable object detection algorithm for exploiting semantic information from the incoming frames. These outputs are integrated into geometric regions of the 3D map, which are carried by the geometric-based incremental segmentation method. The strategy of assigning class probabilities to each segmented region, not each element (e.g., surfels and voxels), notably reduces the computational cost, as well as the memory footprint. In addition to efficiency, by geometrically segmenting the 3D map first, clear boundaries between objects appear. We complementarily improve the geometric-based segmentation results beyond the geometric only to the semantic-aware representation. We validate the proposed method's accuracy and computational efficiency through experiments in a common office scene.

AB - Incrementally, building a 3D map in which object instances are semantically annotated has a wide range of applications, including scene understanding, human-robot interactions, and simultaneous localization and mapping extensions. Although researchers are developing efficient and accurate systems, these methods still face a critical issue: Real-time processing, because the task requires a series of heavy processing components, e.g., camera pose estimation, 3D map reconstruction, and especially recognition. In this paper, we propose a novel object-oriented semantic mapping approach aiming at overcoming such issues by introducing highly accurate object-oriented semantic scene reconstruction in real time. For high efficiency, the proposed method employs a fast and scalable object detection algorithm for exploiting semantic information from the incoming frames. These outputs are integrated into geometric regions of the 3D map, which are carried by the geometric-based incremental segmentation method. The strategy of assigning class probabilities to each segmented region, not each element (e.g., surfels and voxels), notably reduces the computational cost, as well as the memory footprint. In addition to efficiency, by geometrically segmenting the 3D map first, clear boundaries between objects appear. We complementarily improve the geometric-based segmentation results beyond the geometric only to the semantic-aware representation. We validate the proposed method's accuracy and computational efficiency through experiments in a common office scene.

KW - object detection

KW - segmentation

KW - Semantic mapping

KW - SLAM

UR - http://www.scopus.com/inward/record.url?scp=85058880696&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85058880696&partnerID=8YFLogxK

U2 - 10.1109/ACCESS.2018.2887022

DO - 10.1109/ACCESS.2018.2887022

M3 - Article

VL - 7

SP - 3206

EP - 3213

JO - IEEE Access

JF - IEEE Access

SN - 2169-3536

M1 - 8579143

ER -