A novel inpainting-based layered depth video for 3DTV

Ismaël Daribo, Hideo Saito

Research output: Contribution to journalArticle

74 Citations (Scopus)

Abstract

Layered Depth Video (LDV) is recognized as a promising 3D video data representation for supporting advanced 3D video services required in Multiview Video (MVV) systems such as Three-Dimensional Television (3DTV). This representation consists of one full or central view in the form of a video-plus-depth sequence as the main layer, and additional enhancement layers including residual texture and depth data that represent side views. LDV is thus both a derivative of and an alternative to Multiview Video-plus-Depth (MVD) representation by only transmit one full view with associated residual data over the channel. There is a risk, however, of residual data information rapidly increasing as the distance between the center view and side views increases. This occurs when parts of the central view are not visible in the side views, leaving blank spots called disocclusions. These disocclusions may grow larger, which increases the amount of residual data that needs to be sent with the main layer. In this paper, we address the residual layer generation problem. We propose an inpainting-based LDV generation method to reduce the amount of residual data to send by retrieving the missing pixels from the main layer. In the proposed method, we take into account the depth information by distinguishing between foreground and background parts of the scene, at low complexity, during the texture and structure propagation stage of the inpainting process. Experimental results demonstrated the effectiveness of the proposed method.

Original languageEnglish
Article number5743033
Pages (from-to)533-541
Number of pages9
JournalIEEE Transactions on Broadcasting
Volume57
Issue number2 PART 2
DOIs
Publication statusPublished - 2011 Jun

Fingerprint

Textures
Television
Pixels
Derivatives

Keywords

  • 3DTV
  • DIBR
  • inpainting
  • layered depth video

ASJC Scopus subject areas

  • Electrical and Electronic Engineering
  • Media Technology

Cite this

A novel inpainting-based layered depth video for 3DTV. / Daribo, Ismaël; Saito, Hideo.

In: IEEE Transactions on Broadcasting, Vol. 57, No. 2 PART 2, 5743033, 06.2011, p. 533-541.

Research output: Contribution to journalArticle

Daribo, Ismaël ; Saito, Hideo. / A novel inpainting-based layered depth video for 3DTV. In: IEEE Transactions on Broadcasting. 2011 ; Vol. 57, No. 2 PART 2. pp. 533-541.
@article{aef96c1915e1426d882dd78df91b0cc0,
title = "A novel inpainting-based layered depth video for 3DTV",
abstract = "Layered Depth Video (LDV) is recognized as a promising 3D video data representation for supporting advanced 3D video services required in Multiview Video (MVV) systems such as Three-Dimensional Television (3DTV). This representation consists of one full or central view in the form of a video-plus-depth sequence as the main layer, and additional enhancement layers including residual texture and depth data that represent side views. LDV is thus both a derivative of and an alternative to Multiview Video-plus-Depth (MVD) representation by only transmit one full view with associated residual data over the channel. There is a risk, however, of residual data information rapidly increasing as the distance between the center view and side views increases. This occurs when parts of the central view are not visible in the side views, leaving blank spots called disocclusions. These disocclusions may grow larger, which increases the amount of residual data that needs to be sent with the main layer. In this paper, we address the residual layer generation problem. We propose an inpainting-based LDV generation method to reduce the amount of residual data to send by retrieving the missing pixels from the main layer. In the proposed method, we take into account the depth information by distinguishing between foreground and background parts of the scene, at low complexity, during the texture and structure propagation stage of the inpainting process. Experimental results demonstrated the effectiveness of the proposed method.",
keywords = "3DTV, DIBR, inpainting, layered depth video",
author = "Isma{\"e}l Daribo and Hideo Saito",
year = "2011",
month = "6",
doi = "10.1109/TBC.2011.2125110",
language = "English",
volume = "57",
pages = "533--541",
journal = "IEEE Transactions on Broadcasting",
issn = "0018-9316",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
number = "2 PART 2",

}

TY - JOUR

T1 - A novel inpainting-based layered depth video for 3DTV

AU - Daribo, Ismaël

AU - Saito, Hideo

PY - 2011/6

Y1 - 2011/6

N2 - Layered Depth Video (LDV) is recognized as a promising 3D video data representation for supporting advanced 3D video services required in Multiview Video (MVV) systems such as Three-Dimensional Television (3DTV). This representation consists of one full or central view in the form of a video-plus-depth sequence as the main layer, and additional enhancement layers including residual texture and depth data that represent side views. LDV is thus both a derivative of and an alternative to Multiview Video-plus-Depth (MVD) representation by only transmit one full view with associated residual data over the channel. There is a risk, however, of residual data information rapidly increasing as the distance between the center view and side views increases. This occurs when parts of the central view are not visible in the side views, leaving blank spots called disocclusions. These disocclusions may grow larger, which increases the amount of residual data that needs to be sent with the main layer. In this paper, we address the residual layer generation problem. We propose an inpainting-based LDV generation method to reduce the amount of residual data to send by retrieving the missing pixels from the main layer. In the proposed method, we take into account the depth information by distinguishing between foreground and background parts of the scene, at low complexity, during the texture and structure propagation stage of the inpainting process. Experimental results demonstrated the effectiveness of the proposed method.

AB - Layered Depth Video (LDV) is recognized as a promising 3D video data representation for supporting advanced 3D video services required in Multiview Video (MVV) systems such as Three-Dimensional Television (3DTV). This representation consists of one full or central view in the form of a video-plus-depth sequence as the main layer, and additional enhancement layers including residual texture and depth data that represent side views. LDV is thus both a derivative of and an alternative to Multiview Video-plus-Depth (MVD) representation by only transmit one full view with associated residual data over the channel. There is a risk, however, of residual data information rapidly increasing as the distance between the center view and side views increases. This occurs when parts of the central view are not visible in the side views, leaving blank spots called disocclusions. These disocclusions may grow larger, which increases the amount of residual data that needs to be sent with the main layer. In this paper, we address the residual layer generation problem. We propose an inpainting-based LDV generation method to reduce the amount of residual data to send by retrieving the missing pixels from the main layer. In the proposed method, we take into account the depth information by distinguishing between foreground and background parts of the scene, at low complexity, during the texture and structure propagation stage of the inpainting process. Experimental results demonstrated the effectiveness of the proposed method.

KW - 3DTV

KW - DIBR

KW - inpainting

KW - layered depth video

UR - http://www.scopus.com/inward/record.url?scp=79957596546&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=79957596546&partnerID=8YFLogxK

U2 - 10.1109/TBC.2011.2125110

DO - 10.1109/TBC.2011.2125110

M3 - Article

VL - 57

SP - 533

EP - 541

JO - IEEE Transactions on Broadcasting

JF - IEEE Transactions on Broadcasting

SN - 0018-9316

IS - 2 PART 2

M1 - 5743033

ER -