Appearance-based virtual view generation from multicamera videos captured in the 3-D room

Hideo Saito, Shigeyuki Baba, Takeo Kanade

Research output: Contribution to journalArticle

35 Citations (Scopus)

Abstract

We present an appearance-based virtual view generation method that allows viewers to fly through a real dynamic scene. The scene is captured by multiple synchronized cameras. Arbitrary views are generated by interpolating two original camera-views near the given viewpoint. The quality of the generated synthetic view is determined by the precision, consistency and density of correspondences between the two images. All or most of previous work that uses interpolation extracts the correspondences from these two images. However, not only is it difficult to do so reliably (the task requires a good stereo algorithm), but also the two images alone sometimes do not have enough information, due to problems such as occlusion. Instead, we take advantage of the fact that we have many views, from which we can extract much more reliable and comprehensive three-dimensional (3-D) geometry of the scene as a 3-D model. Dense and precise correspondences between the two images, to be used for interpolation, are obtained using this constructed 3-D model. Pseudo correspondences are even obtained for regions occluded in one of the cameras and then we used to correctly interpolate between the two images. Our method of 3-D modeling from multiple images uses the Multiple Baseline Stereo method and the Shape from Silhoutte method. The virtual view sequences are presented for demonstrating the performance of the virtual view generation in the 3-D Room.

Original languageEnglish
Pages (from-to)303-316
Number of pages14
JournalIEEE Transactions on Multimedia
Volume5
Issue number3
DOIs
Publication statusPublished - 2003 Sep 1

Keywords

  • 3-D model
  • Image based rendering
  • Model based rendering
  • Multibaseline stereo
  • Multiple-view images
  • Shape from silhouette

ASJC Scopus subject areas

  • Signal Processing
  • Media Technology
  • Computer Science Applications
  • Electrical and Electronic Engineering

Fingerprint Dive into the research topics of 'Appearance-based virtual view generation from multicamera videos captured in the 3-D room'. Together they form a unique fingerprint.

  • Cite this