TY - JOUR
T1 - Prakash
T2 - Lighting aware motion capture using photosensing markers and multiplexed illuminators
AU - Raskar, Ramesh
AU - Nii, Hideaki
AU - Dedecker, Bert
AU - Hashimoto, Yuki
AU - Summet, Jay
AU - Moore, Dylan
AU - Zhao, Yong
AU - Westhues, Jonathan
AU - Dietz, Paul
AU - Barnwell, John
AU - Nayar, Shree
AU - Inami, Masahiko
AU - Bekaert, Philippe
AU - Noland, Michael
AU - Branzoi, Vlad
AU - Bruns, Erich
PY - 2007/7/29
Y1 - 2007/7/29
N2 - In this paper, we present a high speed optical motion capture method that can measure three dimensional motion, orientation, and incident illumination at tagged points in a scene. We use tracking tags that work in natural lighting conditions and can be imperceptibly embedded in attire or other objects. Our system supports an unlimited number of tags in a scene, with each tag uniquely identified to eliminate marker reacquisition issues. Our tags also provide incident illumination data which can be used to match scene lighting when inserting synthetic elements. The technique is therefore ideal for on-set motion capture or real-time broadcasting of virtual sets. Unlike previous methods that employ high speed cameras or scanning lasers, we capture the scene appearance using the simplest possible optical devices - a light-emitting diode (LED) with a passive binary mask used as the transmitter and a photosensor used as the receiver. We strategically place a set of optical transmitters to spatio-temporally encode the volume of interest. Photosensors attached to scene points demultiplex the coded optical signals from multiple transmitters, allowing us to compute not only receiver location and orientation but also their incident illumination and the reflectance of the surfaces to which the photosensors are attached. We use our untethered tag system, called Prakash, to demonstrate methods of adding special effects to captured videos that cannot be accomplished using pure vision techniques that rely on camera images.
AB - In this paper, we present a high speed optical motion capture method that can measure three dimensional motion, orientation, and incident illumination at tagged points in a scene. We use tracking tags that work in natural lighting conditions and can be imperceptibly embedded in attire or other objects. Our system supports an unlimited number of tags in a scene, with each tag uniquely identified to eliminate marker reacquisition issues. Our tags also provide incident illumination data which can be used to match scene lighting when inserting synthetic elements. The technique is therefore ideal for on-set motion capture or real-time broadcasting of virtual sets. Unlike previous methods that employ high speed cameras or scanning lasers, we capture the scene appearance using the simplest possible optical devices - a light-emitting diode (LED) with a passive binary mask used as the transmitter and a photosensor used as the receiver. We strategically place a set of optical transmitters to spatio-temporally encode the volume of interest. Photosensors attached to scene points demultiplex the coded optical signals from multiple transmitters, allowing us to compute not only receiver location and orientation but also their incident illumination and the reflectance of the surfaces to which the photosensors are attached. We use our untethered tag system, called Prakash, to demonstrate methods of adding special effects to captured videos that cannot be accomplished using pure vision techniques that rely on camera images.
UR - http://www.scopus.com/inward/record.url?scp=34547691016&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=34547691016&partnerID=8YFLogxK
U2 - 10.1145/1276377.1276422
DO - 10.1145/1276377.1276422
M3 - Article
AN - SCOPUS:34547691016
SN - 0730-0301
VL - 26
JO - ACM Transactions on Graphics
JF - ACM Transactions on Graphics
IS - 3
M1 - 1276422
ER -