Augmented reality for 3D TV using depth camera input

François De Sorbier, Yuki Takaya, Yuko Uematsu, Ismael Daribo, Hideo Saito

Research output: Chapter in Book/Report/Conference proceedingConference contribution

5 Citations (Scopus)

Abstract

This paper presents a capture system based on a depth camera that is used for an augmented reality application. Most of depth cameras are unable to capture the color information corresponding to the same viewpoint. A color is then added besides the depth camera and we applied a transformation algorithm to match the depth map with the color camera's viewpoint. Then, using a depth image based rendering (DIBR) approaches, it becomes possible to synthesize new virtual views from the 2D-plus-depth data. We also address some research issues in the generation of the virtual view is to deal with the newly exposed areas, appearing as holes and denoted as occlusions, which may be revealed in each warped image. The color image and its corresponding enhanced depth image are then combined to produce a mesh representing the real scene. It is used to easily integrate virtual objects in the real scene. Finally, the result can be rendered to create the input image required by a auto-stereoscopic screen.

Original languageEnglish
Title of host publication2010 16th International Conference on Virtual Systems and Multimedia, VSMM 2010
Pages117-123
Number of pages7
DOIs
Publication statusPublished - 2010
Event2010 16th International Conference on Virtual Systems and Multimedia, VSMM 2010 - Seoul, Korea, Republic of
Duration: 2010 Oct 202010 Oct 23

Other

Other2010 16th International Conference on Virtual Systems and Multimedia, VSMM 2010
CountryKorea, Republic of
CitySeoul
Period10/10/2010/10/23

Fingerprint

Augmented reality
Cameras
Color

Keywords

  • 3D TV
  • Augmented reality
  • Auto-stereoscopic display
  • Depth image based rendering
  • Depth video

ASJC Scopus subject areas

  • Artificial Intelligence
  • Computer Networks and Communications
  • Human-Computer Interaction

Cite this

De Sorbier, F., Takaya, Y., Uematsu, Y., Daribo, I., & Saito, H. (2010). Augmented reality for 3D TV using depth camera input. In 2010 16th International Conference on Virtual Systems and Multimedia, VSMM 2010 (pp. 117-123). [5665956] https://doi.org/10.1109/VSMM.2010.5665956

Augmented reality for 3D TV using depth camera input. / De Sorbier, François; Takaya, Yuki; Uematsu, Yuko; Daribo, Ismael; Saito, Hideo.

2010 16th International Conference on Virtual Systems and Multimedia, VSMM 2010. 2010. p. 117-123 5665956.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

De Sorbier, F, Takaya, Y, Uematsu, Y, Daribo, I & Saito, H 2010, Augmented reality for 3D TV using depth camera input. in 2010 16th International Conference on Virtual Systems and Multimedia, VSMM 2010., 5665956, pp. 117-123, 2010 16th International Conference on Virtual Systems and Multimedia, VSMM 2010, Seoul, Korea, Republic of, 10/10/20. https://doi.org/10.1109/VSMM.2010.5665956
De Sorbier F, Takaya Y, Uematsu Y, Daribo I, Saito H. Augmented reality for 3D TV using depth camera input. In 2010 16th International Conference on Virtual Systems and Multimedia, VSMM 2010. 2010. p. 117-123. 5665956 https://doi.org/10.1109/VSMM.2010.5665956
De Sorbier, François ; Takaya, Yuki ; Uematsu, Yuko ; Daribo, Ismael ; Saito, Hideo. / Augmented reality for 3D TV using depth camera input. 2010 16th International Conference on Virtual Systems and Multimedia, VSMM 2010. 2010. pp. 117-123
@inproceedings{d2f43e8feb664c20b2647cdca8b6f636,
title = "Augmented reality for 3D TV using depth camera input",
abstract = "This paper presents a capture system based on a depth camera that is used for an augmented reality application. Most of depth cameras are unable to capture the color information corresponding to the same viewpoint. A color is then added besides the depth camera and we applied a transformation algorithm to match the depth map with the color camera's viewpoint. Then, using a depth image based rendering (DIBR) approaches, it becomes possible to synthesize new virtual views from the 2D-plus-depth data. We also address some research issues in the generation of the virtual view is to deal with the newly exposed areas, appearing as holes and denoted as occlusions, which may be revealed in each warped image. The color image and its corresponding enhanced depth image are then combined to produce a mesh representing the real scene. It is used to easily integrate virtual objects in the real scene. Finally, the result can be rendered to create the input image required by a auto-stereoscopic screen.",
keywords = "3D TV, Augmented reality, Auto-stereoscopic display, Depth image based rendering, Depth video",
author = "{De Sorbier}, Fran{\cc}ois and Yuki Takaya and Yuko Uematsu and Ismael Daribo and Hideo Saito",
year = "2010",
doi = "10.1109/VSMM.2010.5665956",
language = "English",
isbn = "9781424490264",
pages = "117--123",
booktitle = "2010 16th International Conference on Virtual Systems and Multimedia, VSMM 2010",

}

TY - GEN

T1 - Augmented reality for 3D TV using depth camera input

AU - De Sorbier, François

AU - Takaya, Yuki

AU - Uematsu, Yuko

AU - Daribo, Ismael

AU - Saito, Hideo

PY - 2010

Y1 - 2010

N2 - This paper presents a capture system based on a depth camera that is used for an augmented reality application. Most of depth cameras are unable to capture the color information corresponding to the same viewpoint. A color is then added besides the depth camera and we applied a transformation algorithm to match the depth map with the color camera's viewpoint. Then, using a depth image based rendering (DIBR) approaches, it becomes possible to synthesize new virtual views from the 2D-plus-depth data. We also address some research issues in the generation of the virtual view is to deal with the newly exposed areas, appearing as holes and denoted as occlusions, which may be revealed in each warped image. The color image and its corresponding enhanced depth image are then combined to produce a mesh representing the real scene. It is used to easily integrate virtual objects in the real scene. Finally, the result can be rendered to create the input image required by a auto-stereoscopic screen.

AB - This paper presents a capture system based on a depth camera that is used for an augmented reality application. Most of depth cameras are unable to capture the color information corresponding to the same viewpoint. A color is then added besides the depth camera and we applied a transformation algorithm to match the depth map with the color camera's viewpoint. Then, using a depth image based rendering (DIBR) approaches, it becomes possible to synthesize new virtual views from the 2D-plus-depth data. We also address some research issues in the generation of the virtual view is to deal with the newly exposed areas, appearing as holes and denoted as occlusions, which may be revealed in each warped image. The color image and its corresponding enhanced depth image are then combined to produce a mesh representing the real scene. It is used to easily integrate virtual objects in the real scene. Finally, the result can be rendered to create the input image required by a auto-stereoscopic screen.

KW - 3D TV

KW - Augmented reality

KW - Auto-stereoscopic display

KW - Depth image based rendering

KW - Depth video

UR - http://www.scopus.com/inward/record.url?scp=78651450319&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=78651450319&partnerID=8YFLogxK

U2 - 10.1109/VSMM.2010.5665956

DO - 10.1109/VSMM.2010.5665956

M3 - Conference contribution

AN - SCOPUS:78651450319

SN - 9781424490264

SP - 117

EP - 123

BT - 2010 16th International Conference on Virtual Systems and Multimedia, VSMM 2010

ER -