3D semantic segmentation for high-resolution aerial survey derived point clouds using deep learning (demonstration)

Haoyi Xiu, Poliyapram Vinayaraj, Kyoung Sook Kim, Ryosuke Nakamura, Wanglin Yan

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Three-dimensional (3D) Semantic segmentation of aerial derived point cloud aims at assigning each point to a semantic class such as building, tree, road, and so on. Accurate 3D-segmentation results can be used as an essential information for constructing 3D city models, for assessing the urban expansion and economical condition. However, the fine-grained semantic segmentation is a challenge in high-resolution point cloud due to irregularly distributed points unlike regular pixels of image. In this demonstration, we present a case study to apply PointNet, a novel deep learning network, to outdoor aerial survey derived point clouds by considering intensity (depth) as well as spectral information (RGB). PointNet was basically designed for indoor point cloud data based on the permutation invariance of 3D points. We firstly fuse two surveying datasets of Light Detection and ranging (LiDAR) and aerial images for generating multi-sourced aerial point clouds (RGB-DI). Then, each point of fused data is classified into a semantic class of ordinary building, public facility, apartment, factory, transportation network, park, and water by reworking PointNet. The result of our approach by using deep learning shows about 0.88 accuracy and 0.64 F-measure of semantic segmentation with the RGB-DI data we have fused. It outperforms a Support Vector Machine(SVM) approach based on geometric features of linearity, planarity, scattering, and verticality of a set of 3D points.

Original languageEnglish
Title of host publication26th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems, ACM SIGSPATIAL GIS 2018
EditorsLi Xiong, Roberto Tamassia, Kashani Farnoush Banaei, Ralf Hartmut Guting, Erik Hoel
PublisherAssociation for Computing Machinery
Pages588-591
Number of pages4
ISBN (Electronic)9781450358897
DOIs
Publication statusPublished - 2018 Nov 6
Event26th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems, ACM SIGSPATIAL GIS 2018 - Seattle, United States
Duration: 2018 Nov 62018 Nov 9

Other

Other26th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems, ACM SIGSPATIAL GIS 2018
CountryUnited States
CitySeattle
Period18/11/618/11/9

Fingerprint

aerial survey
Point Cloud
segmentation
Demonstrations
High Resolution
Segmentation
learning
Semantics
Antennas
linearity
reworking
Planarity
Aerial Image
Transportation Networks
surveying
Surveying
Electric fuses
pixel
Invariance
Linearity

Keywords

  • 3D-segmentation
  • Aerial images
  • Deep learning
  • Point cloud
  • PointNet

ASJC Scopus subject areas

  • Earth-Surface Processes
  • Computer Science Applications
  • Modelling and Simulation
  • Computer Graphics and Computer-Aided Design
  • Information Systems

Cite this

Xiu, H., Vinayaraj, P., Kim, K. S., Nakamura, R., & Yan, W. (2018). 3D semantic segmentation for high-resolution aerial survey derived point clouds using deep learning (demonstration). In L. Xiong, R. Tamassia, K. F. Banaei, R. H. Guting, & E. Hoel (Eds.), 26th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems, ACM SIGSPATIAL GIS 2018 (pp. 588-591). Association for Computing Machinery. https://doi.org/10.1145/3274895.3274950

3D semantic segmentation for high-resolution aerial survey derived point clouds using deep learning (demonstration). / Xiu, Haoyi; Vinayaraj, Poliyapram; Kim, Kyoung Sook; Nakamura, Ryosuke; Yan, Wanglin.

26th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems, ACM SIGSPATIAL GIS 2018. ed. / Li Xiong; Roberto Tamassia; Kashani Farnoush Banaei; Ralf Hartmut Guting; Erik Hoel. Association for Computing Machinery, 2018. p. 588-591.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Xiu, H, Vinayaraj, P, Kim, KS, Nakamura, R & Yan, W 2018, 3D semantic segmentation for high-resolution aerial survey derived point clouds using deep learning (demonstration). in L Xiong, R Tamassia, KF Banaei, RH Guting & E Hoel (eds), 26th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems, ACM SIGSPATIAL GIS 2018. Association for Computing Machinery, pp. 588-591, 26th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems, ACM SIGSPATIAL GIS 2018, Seattle, United States, 18/11/6. https://doi.org/10.1145/3274895.3274950
Xiu H, Vinayaraj P, Kim KS, Nakamura R, Yan W. 3D semantic segmentation for high-resolution aerial survey derived point clouds using deep learning (demonstration). In Xiong L, Tamassia R, Banaei KF, Guting RH, Hoel E, editors, 26th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems, ACM SIGSPATIAL GIS 2018. Association for Computing Machinery. 2018. p. 588-591 https://doi.org/10.1145/3274895.3274950
Xiu, Haoyi ; Vinayaraj, Poliyapram ; Kim, Kyoung Sook ; Nakamura, Ryosuke ; Yan, Wanglin. / 3D semantic segmentation for high-resolution aerial survey derived point clouds using deep learning (demonstration). 26th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems, ACM SIGSPATIAL GIS 2018. editor / Li Xiong ; Roberto Tamassia ; Kashani Farnoush Banaei ; Ralf Hartmut Guting ; Erik Hoel. Association for Computing Machinery, 2018. pp. 588-591
@inproceedings{70d157de12d142959fbe1a36cdeb9c03,
title = "3D semantic segmentation for high-resolution aerial survey derived point clouds using deep learning (demonstration)",
abstract = "Three-dimensional (3D) Semantic segmentation of aerial derived point cloud aims at assigning each point to a semantic class such as building, tree, road, and so on. Accurate 3D-segmentation results can be used as an essential information for constructing 3D city models, for assessing the urban expansion and economical condition. However, the fine-grained semantic segmentation is a challenge in high-resolution point cloud due to irregularly distributed points unlike regular pixels of image. In this demonstration, we present a case study to apply PointNet, a novel deep learning network, to outdoor aerial survey derived point clouds by considering intensity (depth) as well as spectral information (RGB). PointNet was basically designed for indoor point cloud data based on the permutation invariance of 3D points. We firstly fuse two surveying datasets of Light Detection and ranging (LiDAR) and aerial images for generating multi-sourced aerial point clouds (RGB-DI). Then, each point of fused data is classified into a semantic class of ordinary building, public facility, apartment, factory, transportation network, park, and water by reworking PointNet. The result of our approach by using deep learning shows about 0.88 accuracy and 0.64 F-measure of semantic segmentation with the RGB-DI data we have fused. It outperforms a Support Vector Machine(SVM) approach based on geometric features of linearity, planarity, scattering, and verticality of a set of 3D points.",
keywords = "3D-segmentation, Aerial images, Deep learning, Point cloud, PointNet",
author = "Haoyi Xiu and Poliyapram Vinayaraj and Kim, {Kyoung Sook} and Ryosuke Nakamura and Wanglin Yan",
year = "2018",
month = "11",
day = "6",
doi = "10.1145/3274895.3274950",
language = "English",
pages = "588--591",
editor = "Li Xiong and Roberto Tamassia and Banaei, {Kashani Farnoush} and Guting, {Ralf Hartmut} and Erik Hoel",
booktitle = "26th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems, ACM SIGSPATIAL GIS 2018",
publisher = "Association for Computing Machinery",

}

TY - GEN

T1 - 3D semantic segmentation for high-resolution aerial survey derived point clouds using deep learning (demonstration)

AU - Xiu, Haoyi

AU - Vinayaraj, Poliyapram

AU - Kim, Kyoung Sook

AU - Nakamura, Ryosuke

AU - Yan, Wanglin

PY - 2018/11/6

Y1 - 2018/11/6

N2 - Three-dimensional (3D) Semantic segmentation of aerial derived point cloud aims at assigning each point to a semantic class such as building, tree, road, and so on. Accurate 3D-segmentation results can be used as an essential information for constructing 3D city models, for assessing the urban expansion and economical condition. However, the fine-grained semantic segmentation is a challenge in high-resolution point cloud due to irregularly distributed points unlike regular pixels of image. In this demonstration, we present a case study to apply PointNet, a novel deep learning network, to outdoor aerial survey derived point clouds by considering intensity (depth) as well as spectral information (RGB). PointNet was basically designed for indoor point cloud data based on the permutation invariance of 3D points. We firstly fuse two surveying datasets of Light Detection and ranging (LiDAR) and aerial images for generating multi-sourced aerial point clouds (RGB-DI). Then, each point of fused data is classified into a semantic class of ordinary building, public facility, apartment, factory, transportation network, park, and water by reworking PointNet. The result of our approach by using deep learning shows about 0.88 accuracy and 0.64 F-measure of semantic segmentation with the RGB-DI data we have fused. It outperforms a Support Vector Machine(SVM) approach based on geometric features of linearity, planarity, scattering, and verticality of a set of 3D points.

AB - Three-dimensional (3D) Semantic segmentation of aerial derived point cloud aims at assigning each point to a semantic class such as building, tree, road, and so on. Accurate 3D-segmentation results can be used as an essential information for constructing 3D city models, for assessing the urban expansion and economical condition. However, the fine-grained semantic segmentation is a challenge in high-resolution point cloud due to irregularly distributed points unlike regular pixels of image. In this demonstration, we present a case study to apply PointNet, a novel deep learning network, to outdoor aerial survey derived point clouds by considering intensity (depth) as well as spectral information (RGB). PointNet was basically designed for indoor point cloud data based on the permutation invariance of 3D points. We firstly fuse two surveying datasets of Light Detection and ranging (LiDAR) and aerial images for generating multi-sourced aerial point clouds (RGB-DI). Then, each point of fused data is classified into a semantic class of ordinary building, public facility, apartment, factory, transportation network, park, and water by reworking PointNet. The result of our approach by using deep learning shows about 0.88 accuracy and 0.64 F-measure of semantic segmentation with the RGB-DI data we have fused. It outperforms a Support Vector Machine(SVM) approach based on geometric features of linearity, planarity, scattering, and verticality of a set of 3D points.

KW - 3D-segmentation

KW - Aerial images

KW - Deep learning

KW - Point cloud

KW - PointNet

UR - http://www.scopus.com/inward/record.url?scp=85058658588&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85058658588&partnerID=8YFLogxK

U2 - 10.1145/3274895.3274950

DO - 10.1145/3274895.3274950

M3 - Conference contribution

SP - 588

EP - 591

BT - 26th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems, ACM SIGSPATIAL GIS 2018

A2 - Xiong, Li

A2 - Tamassia, Roberto

A2 - Banaei, Kashani Farnoush

A2 - Guting, Ralf Hartmut

A2 - Hoel, Erik

PB - Association for Computing Machinery

ER -