Improving quality of free-viewpoint image by mesh based 3D shape deformation

Satoshi Yaguchi, Hideo Saito

Research output: Chapter in Book/Report/Conference proceedingConference contribution

3 Citations (Scopus)

Abstract

In this paper, we present a method to synthesize high-quality virtual viewpoint image targeting the detailed texture objects. About 30 images are taken from multiple uncalibrated cameras around the object, and the Visual Hull model is reconstructed with Shape from Silhouette method. To deform 3D surface model that is converted from Visual Hull Model using the information such as image texture and object silhouette, the difference between the real object and the reconstructed model is evaluated as a cost function of optimization problem. Our deforming model algorithm is based on single vertex iterative shifting. The vertex of surface triangle mesh is moved to the selected candidate point that maximizes the cost function. The cost function is consisted by four constraint criteria, texture correlation, smoothness, object silhouette, and mesh shape regularity. In addition to the cost function, such as judging mesh direction and combining / dividing meshes are applied for refined 3D models to avoid mesh folding and mesh size unevenness. The refined model provides a quite accurate dense corresponding relationship between the input images, so that high quality image can be synthesized at virtual viewpoint. We also demonstrate the proposed method by showing virtual viewpoint images to applying the real image that are taken from multiple uncalibrated cameras.

Original languageEnglish
Title of host publication14th International Conference in Central Europe on Computer Graphics, Visualization and Computer Vision 2006, WSCG'2006 - In Co-operation with EUROGRAPHICS, Full Papers Proceedings
Pages57-64
Number of pages8
Publication statusPublished - 2006
Event14th International Conference in Central Europe on Computer Graphics, Visualization and Computer Vision 2006, WSCG'2006 - In Co-operation with EUROGRAPHICS - Plzen, Czech Republic
Duration: 2006 Jan 312006 Feb 2

Other

Other14th International Conference in Central Europe on Computer Graphics, Visualization and Computer Vision 2006, WSCG'2006 - In Co-operation with EUROGRAPHICS
CountryCzech Republic
CityPlzen
Period06/1/3106/2/2

Fingerprint

Cost functions
Textures
Cameras
Image texture
Image quality

Keywords

  • Image Based Rendering
  • Shape refinement
  • Shape-from-Silhouette
  • The Visual Hull
  • Weakly calibrated multiple camera system

ASJC Scopus subject areas

  • Computer Graphics and Computer-Aided Design
  • Computer Vision and Pattern Recognition

Cite this

Yaguchi, S., & Saito, H. (2006). Improving quality of free-viewpoint image by mesh based 3D shape deformation. In 14th International Conference in Central Europe on Computer Graphics, Visualization and Computer Vision 2006, WSCG'2006 - In Co-operation with EUROGRAPHICS, Full Papers Proceedings (pp. 57-64)

Improving quality of free-viewpoint image by mesh based 3D shape deformation. / Yaguchi, Satoshi; Saito, Hideo.

14th International Conference in Central Europe on Computer Graphics, Visualization and Computer Vision 2006, WSCG'2006 - In Co-operation with EUROGRAPHICS, Full Papers Proceedings. 2006. p. 57-64.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Yaguchi, S & Saito, H 2006, Improving quality of free-viewpoint image by mesh based 3D shape deformation. in 14th International Conference in Central Europe on Computer Graphics, Visualization and Computer Vision 2006, WSCG'2006 - In Co-operation with EUROGRAPHICS, Full Papers Proceedings. pp. 57-64, 14th International Conference in Central Europe on Computer Graphics, Visualization and Computer Vision 2006, WSCG'2006 - In Co-operation with EUROGRAPHICS, Plzen, Czech Republic, 06/1/31.
Yaguchi S, Saito H. Improving quality of free-viewpoint image by mesh based 3D shape deformation. In 14th International Conference in Central Europe on Computer Graphics, Visualization and Computer Vision 2006, WSCG'2006 - In Co-operation with EUROGRAPHICS, Full Papers Proceedings. 2006. p. 57-64
Yaguchi, Satoshi ; Saito, Hideo. / Improving quality of free-viewpoint image by mesh based 3D shape deformation. 14th International Conference in Central Europe on Computer Graphics, Visualization and Computer Vision 2006, WSCG'2006 - In Co-operation with EUROGRAPHICS, Full Papers Proceedings. 2006. pp. 57-64
@inproceedings{05b47c07d45e449e895959d9b5f05bb6,
title = "Improving quality of free-viewpoint image by mesh based 3D shape deformation",
abstract = "In this paper, we present a method to synthesize high-quality virtual viewpoint image targeting the detailed texture objects. About 30 images are taken from multiple uncalibrated cameras around the object, and the Visual Hull model is reconstructed with Shape from Silhouette method. To deform 3D surface model that is converted from Visual Hull Model using the information such as image texture and object silhouette, the difference between the real object and the reconstructed model is evaluated as a cost function of optimization problem. Our deforming model algorithm is based on single vertex iterative shifting. The vertex of surface triangle mesh is moved to the selected candidate point that maximizes the cost function. The cost function is consisted by four constraint criteria, texture correlation, smoothness, object silhouette, and mesh shape regularity. In addition to the cost function, such as judging mesh direction and combining / dividing meshes are applied for refined 3D models to avoid mesh folding and mesh size unevenness. The refined model provides a quite accurate dense corresponding relationship between the input images, so that high quality image can be synthesized at virtual viewpoint. We also demonstrate the proposed method by showing virtual viewpoint images to applying the real image that are taken from multiple uncalibrated cameras.",
keywords = "Image Based Rendering, Shape refinement, Shape-from-Silhouette, The Visual Hull, Weakly calibrated multiple camera system",
author = "Satoshi Yaguchi and Hideo Saito",
year = "2006",
language = "English",
isbn = "8086943038",
pages = "57--64",
booktitle = "14th International Conference in Central Europe on Computer Graphics, Visualization and Computer Vision 2006, WSCG'2006 - In Co-operation with EUROGRAPHICS, Full Papers Proceedings",

}

TY - GEN

T1 - Improving quality of free-viewpoint image by mesh based 3D shape deformation

AU - Yaguchi, Satoshi

AU - Saito, Hideo

PY - 2006

Y1 - 2006

N2 - In this paper, we present a method to synthesize high-quality virtual viewpoint image targeting the detailed texture objects. About 30 images are taken from multiple uncalibrated cameras around the object, and the Visual Hull model is reconstructed with Shape from Silhouette method. To deform 3D surface model that is converted from Visual Hull Model using the information such as image texture and object silhouette, the difference between the real object and the reconstructed model is evaluated as a cost function of optimization problem. Our deforming model algorithm is based on single vertex iterative shifting. The vertex of surface triangle mesh is moved to the selected candidate point that maximizes the cost function. The cost function is consisted by four constraint criteria, texture correlation, smoothness, object silhouette, and mesh shape regularity. In addition to the cost function, such as judging mesh direction and combining / dividing meshes are applied for refined 3D models to avoid mesh folding and mesh size unevenness. The refined model provides a quite accurate dense corresponding relationship between the input images, so that high quality image can be synthesized at virtual viewpoint. We also demonstrate the proposed method by showing virtual viewpoint images to applying the real image that are taken from multiple uncalibrated cameras.

AB - In this paper, we present a method to synthesize high-quality virtual viewpoint image targeting the detailed texture objects. About 30 images are taken from multiple uncalibrated cameras around the object, and the Visual Hull model is reconstructed with Shape from Silhouette method. To deform 3D surface model that is converted from Visual Hull Model using the information such as image texture and object silhouette, the difference between the real object and the reconstructed model is evaluated as a cost function of optimization problem. Our deforming model algorithm is based on single vertex iterative shifting. The vertex of surface triangle mesh is moved to the selected candidate point that maximizes the cost function. The cost function is consisted by four constraint criteria, texture correlation, smoothness, object silhouette, and mesh shape regularity. In addition to the cost function, such as judging mesh direction and combining / dividing meshes are applied for refined 3D models to avoid mesh folding and mesh size unevenness. The refined model provides a quite accurate dense corresponding relationship between the input images, so that high quality image can be synthesized at virtual viewpoint. We also demonstrate the proposed method by showing virtual viewpoint images to applying the real image that are taken from multiple uncalibrated cameras.

KW - Image Based Rendering

KW - Shape refinement

KW - Shape-from-Silhouette

KW - The Visual Hull

KW - Weakly calibrated multiple camera system

UR - http://www.scopus.com/inward/record.url?scp=58549116845&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=58549116845&partnerID=8YFLogxK

M3 - Conference contribution

SN - 8086943038

SN - 9788086943039

SP - 57

EP - 64

BT - 14th International Conference in Central Europe on Computer Graphics, Visualization and Computer Vision 2006, WSCG'2006 - In Co-operation with EUROGRAPHICS, Full Papers Proceedings

ER -