Arbitrary viewpoint image synthesis based on light field construction from uncalibrated image sequence

Kenkichi Kobayashi, Hideo Saito

Research output: Contribution to journalArticle

Abstract

Recently, image-based rendering in which the actual image is efficiently utilized to synthesize a new image has been attracting attention. There have been many studies of arbitrary viewpoint image synthesis based on the light field, which is an image-based rendering procedure. It is now possible to synthesize images with a high degree of realism. However, in the method based on the theory of the light field, the camera parameters used in the imaging must be known (with calibration). In cases such as imaging by a free hand-held camera, the theory of the light field cannot be applied to the sequence of uncalibrated images. Consequently, this paper proposes an image generation method using the theory of the light field, which can be applied to a sequence of uncalibrated images. In the proposed method, the ray in each frame can be handled appropriately, using only the correspondence between frames, which is derived by tracking the feature points in the uncalibrated image sequence. This is possible because a special space, the projective grid space, which is determined by the two cameras, rather than Euclidean space, that is, ordinary three-dimensional space, is used for light field construction in the proposed method. In the projective grid space, the general geometrical relations among straight lines and planes in Euclidean space do not apply, but the position of the ray in space can be calculated by using the fundamental matrix. The proposed method is applied to uncalibrated image sequences taken by free hand-held video cameras. In experiments, the image from an arbitrary viewpoint is generated in a satisfactory way, indicating the usefulness of the proposed method.

Original languageEnglish
Pages (from-to)11-21
Number of pages11
JournalElectronics and Communications in Japan, Part III: Fundamental Electronic Science (English translation of Denshi Tsushin Gakkai Ronbunshi)
Volume87
Issue number9
DOIs
Publication statusPublished - 2004 Sep 1

Keywords

  • Arbitrary viewpoint image synthesis
  • Epipolar geometry
  • Image-based rendering
  • Light field
  • Self-calibration

ASJC Scopus subject areas

  • Electrical and Electronic Engineering

Fingerprint Dive into the research topics of 'Arbitrary viewpoint image synthesis based on light field construction from uncalibrated image sequence'. Together they form a unique fingerprint.

  • Cite this