Incremental class discovery for semantic segmentation with rgbd sensing

Yoshikatsu Nakajima, Byeongkeun Kang, Hideo Saito, Kris Kitani

研究成果: Conference contribution

4 被引用数 (Scopus)


This work addresses the task of open world semantic segmentation using RGBD sensing to discover new semantic classes over time. Although there are many types of objects in the real-word, current semantic segmentation methods make a closed world assumption and are trained only to segment a limited number of object classes. Towards a more open world approach, we propose a novel method that incrementally learns new classes for image segmentation. The proposed system first segments each RGBD frame using both color and geometric information, and then aggregates that information to build a single segmented dense 3D map of the environment. The segmented 3D map representation is a key component of our approach as it is used to discover new object classes by identifying coherent regions in the 3D map that have no semantic label. The use of coherent region in the 3D map as a primitive element, rather than traditional elements such as surfels or voxels, also significantly reduces the computational complexity and memory use of our method. It thus leads to semi-real-time performance at 10.7 Hz when incrementally updating the dense 3D map at every frame. Through experiments on the NYUDv2 dataset, we demonstrate that the proposed method is able to correctly cluster objects of both known and unseen classes. We also show the quantitative comparison with the state-of-the-art supervised methods, the processing time of each step, and the influences of each component.

ホスト出版物のタイトルProceedings - 2019 International Conference on Computer Vision, ICCV 2019
出版社Institute of Electrical and Electronics Engineers Inc.
出版ステータスPublished - 2019 10月
イベント17th IEEE/CVF International Conference on Computer Vision, ICCV 2019 - Seoul, Korea, Republic of
継続期間: 2019 10月 272019 11月 2


名前Proceedings of the IEEE International Conference on Computer Vision


Conference17th IEEE/CVF International Conference on Computer Vision, ICCV 2019
国/地域Korea, Republic of

ASJC Scopus subject areas

  • ソフトウェア
  • コンピュータ ビジョンおよびパターン認識


「Incremental class discovery for semantic segmentation with rgbd sensing」の研究トピックを掘り下げます。これらがまとまってユニークなフィンガープリントを構成します。