Joint inpainting of RGB and depth images by generative adversarial network with a late fusion approach

Ryo Fujii; Ryo Hachiuma; Hideo Saito

doi:10.1109/ISMAR-Adjunct.2019.00-46

Joint inpainting of RGB and depth images by generative adversarial network with a late fusion approach

Ryo Fujii, Ryo Hachiuma, Hideo Saito

Department of Information and Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

2 Citations (Scopus)

Abstract

Image inpainting aims to restore texture of missing regions in scene from an RGB image. In this paper, we aim to restore not only the texture but also the geometry of the missing regions in scene from a pair of RGB and depth images. Inspired by the recent development of generative adversarial network, we employ an encoder-decoderbased generative adversarial network with the input of RGB and depth image. The experimental results show that our method restores the missing region of both RGB and depth image.

Original language	English
Title of host publication	Adjunct Proceedings of the 2019 IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2019
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	203-204
Number of pages	2
ISBN (Electronic)	9781728147659
DOIs	https://doi.org/10.1109/ISMAR-Adjunct.2019.00-46
Publication status	Published - 2019 Oct
Event	18th IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2019 - Beijing, China Duration: 2019 Oct 14 → 2019 Oct 18

Publication series

Name	Adjunct Proceedings of the 2019 IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2019

Conference

Conference	18th IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2019
Country/Territory	China
City	Beijing
Period	19/10/14 → 19/10/18

Keywords

Artificial-intelligence
Computer-graphics
Computer-vision
Computer-vision-Tasks
Graphics-systems-And-interfaces
Mixed-/-Augmented-reality
Scene-understanding

ASJC Scopus subject areas

Computer Science Applications
Human-Computer Interaction
Media Technology

Access to Document

10.1109/ISMAR-Adjunct.2019.00-46

Cite this

Fujii, R., Hachiuma, R., & Saito, H. (2019). Joint inpainting of RGB and depth images by generative adversarial network with a late fusion approach. In Adjunct Proceedings of the 2019 IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2019 (pp. 203-204). [8951904] (Adjunct Proceedings of the 2019 IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2019). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ISMAR-Adjunct.2019.00-46

Joint inpainting of RGB and depth images by generative adversarial network with a late fusion approach. / Fujii, Ryo; Hachiuma, Ryo; Saito, Hideo.

Adjunct Proceedings of the 2019 IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2019. Institute of Electrical and Electronics Engineers Inc., 2019. p. 203-204 8951904 (Adjunct Proceedings of the 2019 IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2019).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Fujii, R, Hachiuma, R & Saito, H 2019, Joint inpainting of RGB and depth images by generative adversarial network with a late fusion approach. in Adjunct Proceedings of the 2019 IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2019., 8951904, Adjunct Proceedings of the 2019 IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2019, Institute of Electrical and Electronics Engineers Inc., pp. 203-204, 18th IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2019, Beijing, China, 19/10/14. https://doi.org/10.1109/ISMAR-Adjunct.2019.00-46

Fujii R, Hachiuma R, Saito H. Joint inpainting of RGB and depth images by generative adversarial network with a late fusion approach. In Adjunct Proceedings of the 2019 IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2019. Institute of Electrical and Electronics Engineers Inc. 2019. p. 203-204. 8951904. (Adjunct Proceedings of the 2019 IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2019). doi: 10.1109/ISMAR-Adjunct.2019.00-46

Fujii, Ryo ; Hachiuma, Ryo ; Saito, Hideo. / Joint inpainting of RGB and depth images by generative adversarial network with a late fusion approach. Adjunct Proceedings of the 2019 IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2019. Institute of Electrical and Electronics Engineers Inc., 2019. pp. 203-204 (Adjunct Proceedings of the 2019 IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2019).

@inproceedings{b490a1372c9e43acb3bcca221b189b05,

title = "Joint inpainting of RGB and depth images by generative adversarial network with a late fusion approach",

abstract = "Image inpainting aims to restore texture of missing regions in scene from an RGB image. In this paper, we aim to restore not only the texture but also the geometry of the missing regions in scene from a pair of RGB and depth images. Inspired by the recent development of generative adversarial network, we employ an encoder-decoderbased generative adversarial network with the input of RGB and depth image. The experimental results show that our method restores the missing region of both RGB and depth image.",

keywords = "Artificial-intelligence, Computer-graphics, Computer-vision, Computer-vision-Tasks, Graphics-systems-And-interfaces, Mixed-/-Augmented-reality, Scene-understanding",

author = "Ryo Fujii and Ryo Hachiuma and Hideo Saito",

note = "Publisher Copyright: {\textcopyright} 2019 IEEE.; 18th IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2019 ; Conference date: 14-10-2019 Through 18-10-2019",

year = "2019",

month = oct,

doi = "10.1109/ISMAR-Adjunct.2019.00-46",

language = "English",

series = "Adjunct Proceedings of the 2019 IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2019",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "203--204",

booktitle = "Adjunct Proceedings of the 2019 IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2019",

}

TY - GEN

T1 - Joint inpainting of RGB and depth images by generative adversarial network with a late fusion approach

AU - Fujii, Ryo

AU - Hachiuma, Ryo

AU - Saito, Hideo

PY - 2019/10

Y1 - 2019/10

N2 - Image inpainting aims to restore texture of missing regions in scene from an RGB image. In this paper, we aim to restore not only the texture but also the geometry of the missing regions in scene from a pair of RGB and depth images. Inspired by the recent development of generative adversarial network, we employ an encoder-decoderbased generative adversarial network with the input of RGB and depth image. The experimental results show that our method restores the missing region of both RGB and depth image.

AB - Image inpainting aims to restore texture of missing regions in scene from an RGB image. In this paper, we aim to restore not only the texture but also the geometry of the missing regions in scene from a pair of RGB and depth images. Inspired by the recent development of generative adversarial network, we employ an encoder-decoderbased generative adversarial network with the input of RGB and depth image. The experimental results show that our method restores the missing region of both RGB and depth image.

KW - Artificial-intelligence

KW - Computer-graphics

KW - Computer-vision

KW - Computer-vision-Tasks

KW - Graphics-systems-And-interfaces

KW - Mixed-/-Augmented-reality

KW - Scene-understanding

UR - http://www.scopus.com/inward/record.url?scp=85078742050&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85078742050&partnerID=8YFLogxK

U2 - 10.1109/ISMAR-Adjunct.2019.00-46

DO - 10.1109/ISMAR-Adjunct.2019.00-46

M3 - Conference contribution

AN - SCOPUS:85078742050

T3 - Adjunct Proceedings of the 2019 IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2019

SP - 203

EP - 204

BT - Adjunct Proceedings of the 2019 IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2019

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 18th IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2019

Y2 - 14 October 2019 through 18 October 2019

ER -

Joint inpainting of RGB and depth images by generative adversarial network with a late fusion approach

Abstract

Publication series

Conference

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this