Incremental class discovery for semantic segmentation with rgbd sensing

Yoshikatsu Nakajima, Byeongkeun Kang, Hideo Saito, Kris Kitani

研究成果: Conference contribution

抜粋

This work addresses the task of open world semantic segmentation using RGBD sensing to discover new semantic classes over time. Although there are many types of objects in the real-word, current semantic segmentation methods make a closed world assumption and are trained only to segment a limited number of object classes. Towards a more open world approach, we propose a novel method that incrementally learns new classes for image segmentation. The proposed system first segments each RGBD frame using both color and geometric information, and then aggregates that information to build a single segmented dense 3D map of the environment. The segmented 3D map representation is a key component of our approach as it is used to discover new object classes by identifying coherent regions in the 3D map that have no semantic label. The use of coherent region in the 3D map as a primitive element, rather than traditional elements such as surfels or voxels, also significantly reduces the computational complexity and memory use of our method. It thus leads to semi-real-time performance at 10.7 Hz when incrementally updating the dense 3D map at every frame. Through experiments on the NYUDv2 dataset, we demonstrate that the proposed method is able to correctly cluster objects of both known and unseen classes. We also show the quantitative comparison with the state-of-the-art supervised methods, the processing time of each step, and the influences of each component.

元の言語English
ホスト出版物のタイトルProceedings - 2019 International Conference on Computer Vision, ICCV 2019
出版者Institute of Electrical and Electronics Engineers Inc.
ページ972-981
ページ数10
ISBN(電子版)9781728148038
DOI
出版物ステータスPublished - 2019 10
イベント17th IEEE/CVF International Conference on Computer Vision, ICCV 2019 - Seoul, Korea, Republic of
継続期間: 2019 10 272019 11 2

出版物シリーズ

名前Proceedings of the IEEE International Conference on Computer Vision
2019-October
ISSN(印刷物)1550-5499

Conference

Conference17th IEEE/CVF International Conference on Computer Vision, ICCV 2019
Korea, Republic of
Seoul
期間19/10/2719/11/2

ASJC Scopus subject areas

  • Software
  • Computer Vision and Pattern Recognition

フィンガープリント Incremental class discovery for semantic segmentation with rgbd sensing' の研究トピックを掘り下げます。これらはともに一意のフィンガープリントを構成します。

  • これを引用

    Nakajima, Y., Kang, B., Saito, H., & Kitani, K. (2019). Incremental class discovery for semantic segmentation with rgbd sensing. : Proceedings - 2019 International Conference on Computer Vision, ICCV 2019 (pp. 972-981). [9009083] (Proceedings of the IEEE International Conference on Computer Vision; 巻数 2019-October). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICCV.2019.00106