Abstract
We present an appearance-based virtual view generation method that allows viewers to fly through a real dynamic scene. The scene is captured by multiple synchronized cameras. Arbitrary views are generated by interpolating two original camera-views near the given viewpoint. The quality of the generated synthetic view is determined by the precision, consistency and density of correspondences between the two images. All or most of previous work that uses interpolation extracts the correspondences from these two images. However, not only is it difficult to do so reliably (the task requires a good stereo algorithm), but also the two images alone sometimes do not have enough information, due to problems such as occlusion. Instead, we take advantage of the fact that we have many views, from which we can extract much more reliable and comprehensive three-dimensional (3-D) geometry of the scene as a 3-D model. Dense and precise correspondences between the two images, to be used for interpolation, are obtained using this constructed 3-D model. Pseudo correspondences are even obtained for regions occluded in one of the cameras and then we used to correctly interpolate between the two images. Our method of 3-D modeling from multiple images uses the Multiple Baseline Stereo method and the Shape from Silhoutte method. The virtual view sequences are presented for demonstrating the performance of the virtual view generation in the 3-D Room.
| Original language | English |
|---|---|
| Pages (from-to) | 303-316 |
| Number of pages | 14 |
| Journal | IEEE Transactions on Multimedia |
| Volume | 5 |
| Issue number | 3 |
| DOIs | |
| Publication status | Published - 2003 Sep |
Fingerprint
Keywords
- 3-D model
- Image based rendering
- Model based rendering
- Multibaseline stereo
- Multiple-view images
- Shape from silhouette
ASJC Scopus subject areas
- Computer Networks and Communications
- Information Systems
- Computer Graphics and Computer-Aided Design
- Software
Cite this
Appearance-based virtual view generation from multicamera videos captured in the 3-D room. / Saito, Hideo; Baba, Shigeyuki; Kanade, Takeo.
In: IEEE Transactions on Multimedia, Vol. 5, No. 3, 09.2003, p. 303-316.Research output: Contribution to journal › Article
}
TY - JOUR
T1 - Appearance-based virtual view generation from multicamera videos captured in the 3-D room
AU - Saito, Hideo
AU - Baba, Shigeyuki
AU - Kanade, Takeo
PY - 2003/9
Y1 - 2003/9
N2 - We present an appearance-based virtual view generation method that allows viewers to fly through a real dynamic scene. The scene is captured by multiple synchronized cameras. Arbitrary views are generated by interpolating two original camera-views near the given viewpoint. The quality of the generated synthetic view is determined by the precision, consistency and density of correspondences between the two images. All or most of previous work that uses interpolation extracts the correspondences from these two images. However, not only is it difficult to do so reliably (the task requires a good stereo algorithm), but also the two images alone sometimes do not have enough information, due to problems such as occlusion. Instead, we take advantage of the fact that we have many views, from which we can extract much more reliable and comprehensive three-dimensional (3-D) geometry of the scene as a 3-D model. Dense and precise correspondences between the two images, to be used for interpolation, are obtained using this constructed 3-D model. Pseudo correspondences are even obtained for regions occluded in one of the cameras and then we used to correctly interpolate between the two images. Our method of 3-D modeling from multiple images uses the Multiple Baseline Stereo method and the Shape from Silhoutte method. The virtual view sequences are presented for demonstrating the performance of the virtual view generation in the 3-D Room.
AB - We present an appearance-based virtual view generation method that allows viewers to fly through a real dynamic scene. The scene is captured by multiple synchronized cameras. Arbitrary views are generated by interpolating two original camera-views near the given viewpoint. The quality of the generated synthetic view is determined by the precision, consistency and density of correspondences between the two images. All or most of previous work that uses interpolation extracts the correspondences from these two images. However, not only is it difficult to do so reliably (the task requires a good stereo algorithm), but also the two images alone sometimes do not have enough information, due to problems such as occlusion. Instead, we take advantage of the fact that we have many views, from which we can extract much more reliable and comprehensive three-dimensional (3-D) geometry of the scene as a 3-D model. Dense and precise correspondences between the two images, to be used for interpolation, are obtained using this constructed 3-D model. Pseudo correspondences are even obtained for regions occluded in one of the cameras and then we used to correctly interpolate between the two images. Our method of 3-D modeling from multiple images uses the Multiple Baseline Stereo method and the Shape from Silhoutte method. The virtual view sequences are presented for demonstrating the performance of the virtual view generation in the 3-D Room.
KW - 3-D model
KW - Image based rendering
KW - Model based rendering
KW - Multibaseline stereo
KW - Multiple-view images
KW - Shape from silhouette
UR - http://www.scopus.com/inward/record.url?scp=0042913178&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=0042913178&partnerID=8YFLogxK
U2 - 10.1109/TMM.2003.813283
DO - 10.1109/TMM.2003.813283
M3 - Article
AN - SCOPUS:0042913178
VL - 5
SP - 303
EP - 316
JO - IEEE Transactions on Multimedia
JF - IEEE Transactions on Multimedia
SN - 1520-9210
IS - 3
ER -