Prakash

Lighting aware motion capture using photosensing markers and multiplexed illuminators

Ramesh Raskar, Hideaki Nii, Bert Dedecker, Yuki Hashimoto, Jay Summet, Dylan Moore, Yong Zhao, Jonathan Westhues, Paul Dietz, John Barnwell, Shree Nayar, Masahiko Inami, Philippe Bekaert, Michael Noland, Vlad Branzoi, Erich Bruns

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

In this paper, we present a high speed optical motion capture method that can measure three dimensional motion, orientation, and incident illumination at tagged points in a scene. We use tracking tags that work in natural lighting conditions and can be imperceptibly embedded in attire or other objects. Our system supports an unlimited number of tags in a scene, with each tag uniquely identified to eliminate marker reacquisition issues. Our tags also provide incident illumination data which can be used to match scene lighting when inserting synthetic elements. The technique is therefore ideal for on-set motion capture or real-time broadcasting of virtual sets. Unlike previous methods that employ high speed cameras or scanning lasers, we capture the scene appearance using the simplest possible optical devices - a light-emitting diode (LED) with a passive binary mask used as the transmitter and a photosensor used as the receiver. We strategically place a set of optical transmitters to spatio-temporally encode the volume of interest. Photosensors attached to scene points demultiplex the coded optical signals from multiple transmitters, allowing us to compute not only receiver location and orientation but also their incident illumination and the reflectance of the surfaces to which the photosensors are attached. We use our untethered tag system, called Prakash, to demonstrate methods of adding special effects to captured videos that cannot be accomplished using pure vision techniques that rely on camera images.

Original languageEnglish
Title of host publicationProceedings of the ACM SIGGRAPH Conference on Computer Graphics
DOIs
Publication statusPublished - 2007
Externally publishedYes
Event34th Annual Meeting of the Association for Computing Machinery's Special Interest Group on Graphics - San Diego, CA, United States
Duration: 2007 Aug 52007 Aug 9

Other

Other34th Annual Meeting of the Association for Computing Machinery's Special Interest Group on Graphics
CountryUnited States
CitySan Diego, CA
Period07/8/507/8/9

Fingerprint

Lighting
Transmitters
Special effects
High speed cameras
Optical devices
Broadcasting
Light emitting diodes
Masks
Cameras
Scanning
Lasers

ASJC Scopus subject areas

  • Computer Science(all)

Cite this

Raskar, R., Nii, H., Dedecker, B., Hashimoto, Y., Summet, J., Moore, D., ... Bruns, E. (2007). Prakash: Lighting aware motion capture using photosensing markers and multiplexed illuminators. In Proceedings of the ACM SIGGRAPH Conference on Computer Graphics [36] https://doi.org/10.1145/1275808.1276422

Prakash : Lighting aware motion capture using photosensing markers and multiplexed illuminators. / Raskar, Ramesh; Nii, Hideaki; Dedecker, Bert; Hashimoto, Yuki; Summet, Jay; Moore, Dylan; Zhao, Yong; Westhues, Jonathan; Dietz, Paul; Barnwell, John; Nayar, Shree; Inami, Masahiko; Bekaert, Philippe; Noland, Michael; Branzoi, Vlad; Bruns, Erich.

Proceedings of the ACM SIGGRAPH Conference on Computer Graphics. 2007. 36.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Raskar, R, Nii, H, Dedecker, B, Hashimoto, Y, Summet, J, Moore, D, Zhao, Y, Westhues, J, Dietz, P, Barnwell, J, Nayar, S, Inami, M, Bekaert, P, Noland, M, Branzoi, V & Bruns, E 2007, Prakash: Lighting aware motion capture using photosensing markers and multiplexed illuminators. in Proceedings of the ACM SIGGRAPH Conference on Computer Graphics., 36, 34th Annual Meeting of the Association for Computing Machinery's Special Interest Group on Graphics, San Diego, CA, United States, 07/8/5. https://doi.org/10.1145/1275808.1276422
Raskar R, Nii H, Dedecker B, Hashimoto Y, Summet J, Moore D et al. Prakash: Lighting aware motion capture using photosensing markers and multiplexed illuminators. In Proceedings of the ACM SIGGRAPH Conference on Computer Graphics. 2007. 36 https://doi.org/10.1145/1275808.1276422
Raskar, Ramesh ; Nii, Hideaki ; Dedecker, Bert ; Hashimoto, Yuki ; Summet, Jay ; Moore, Dylan ; Zhao, Yong ; Westhues, Jonathan ; Dietz, Paul ; Barnwell, John ; Nayar, Shree ; Inami, Masahiko ; Bekaert, Philippe ; Noland, Michael ; Branzoi, Vlad ; Bruns, Erich. / Prakash : Lighting aware motion capture using photosensing markers and multiplexed illuminators. Proceedings of the ACM SIGGRAPH Conference on Computer Graphics. 2007.
@inproceedings{df4bd34c2b054b9aa8bba6ebc64cb9c3,
title = "Prakash: Lighting aware motion capture using photosensing markers and multiplexed illuminators",
abstract = "In this paper, we present a high speed optical motion capture method that can measure three dimensional motion, orientation, and incident illumination at tagged points in a scene. We use tracking tags that work in natural lighting conditions and can be imperceptibly embedded in attire or other objects. Our system supports an unlimited number of tags in a scene, with each tag uniquely identified to eliminate marker reacquisition issues. Our tags also provide incident illumination data which can be used to match scene lighting when inserting synthetic elements. The technique is therefore ideal for on-set motion capture or real-time broadcasting of virtual sets. Unlike previous methods that employ high speed cameras or scanning lasers, we capture the scene appearance using the simplest possible optical devices - a light-emitting diode (LED) with a passive binary mask used as the transmitter and a photosensor used as the receiver. We strategically place a set of optical transmitters to spatio-temporally encode the volume of interest. Photosensors attached to scene points demultiplex the coded optical signals from multiple transmitters, allowing us to compute not only receiver location and orientation but also their incident illumination and the reflectance of the surfaces to which the photosensors are attached. We use our untethered tag system, called Prakash, to demonstrate methods of adding special effects to captured videos that cannot be accomplished using pure vision techniques that rely on camera images.",
author = "Ramesh Raskar and Hideaki Nii and Bert Dedecker and Yuki Hashimoto and Jay Summet and Dylan Moore and Yong Zhao and Jonathan Westhues and Paul Dietz and John Barnwell and Shree Nayar and Masahiko Inami and Philippe Bekaert and Michael Noland and Vlad Branzoi and Erich Bruns",
year = "2007",
doi = "10.1145/1275808.1276422",
language = "English",
booktitle = "Proceedings of the ACM SIGGRAPH Conference on Computer Graphics",

}

TY - GEN

T1 - Prakash

T2 - Lighting aware motion capture using photosensing markers and multiplexed illuminators

AU - Raskar, Ramesh

AU - Nii, Hideaki

AU - Dedecker, Bert

AU - Hashimoto, Yuki

AU - Summet, Jay

AU - Moore, Dylan

AU - Zhao, Yong

AU - Westhues, Jonathan

AU - Dietz, Paul

AU - Barnwell, John

AU - Nayar, Shree

AU - Inami, Masahiko

AU - Bekaert, Philippe

AU - Noland, Michael

AU - Branzoi, Vlad

AU - Bruns, Erich

PY - 2007

Y1 - 2007

N2 - In this paper, we present a high speed optical motion capture method that can measure three dimensional motion, orientation, and incident illumination at tagged points in a scene. We use tracking tags that work in natural lighting conditions and can be imperceptibly embedded in attire or other objects. Our system supports an unlimited number of tags in a scene, with each tag uniquely identified to eliminate marker reacquisition issues. Our tags also provide incident illumination data which can be used to match scene lighting when inserting synthetic elements. The technique is therefore ideal for on-set motion capture or real-time broadcasting of virtual sets. Unlike previous methods that employ high speed cameras or scanning lasers, we capture the scene appearance using the simplest possible optical devices - a light-emitting diode (LED) with a passive binary mask used as the transmitter and a photosensor used as the receiver. We strategically place a set of optical transmitters to spatio-temporally encode the volume of interest. Photosensors attached to scene points demultiplex the coded optical signals from multiple transmitters, allowing us to compute not only receiver location and orientation but also their incident illumination and the reflectance of the surfaces to which the photosensors are attached. We use our untethered tag system, called Prakash, to demonstrate methods of adding special effects to captured videos that cannot be accomplished using pure vision techniques that rely on camera images.

AB - In this paper, we present a high speed optical motion capture method that can measure three dimensional motion, orientation, and incident illumination at tagged points in a scene. We use tracking tags that work in natural lighting conditions and can be imperceptibly embedded in attire or other objects. Our system supports an unlimited number of tags in a scene, with each tag uniquely identified to eliminate marker reacquisition issues. Our tags also provide incident illumination data which can be used to match scene lighting when inserting synthetic elements. The technique is therefore ideal for on-set motion capture or real-time broadcasting of virtual sets. Unlike previous methods that employ high speed cameras or scanning lasers, we capture the scene appearance using the simplest possible optical devices - a light-emitting diode (LED) with a passive binary mask used as the transmitter and a photosensor used as the receiver. We strategically place a set of optical transmitters to spatio-temporally encode the volume of interest. Photosensors attached to scene points demultiplex the coded optical signals from multiple transmitters, allowing us to compute not only receiver location and orientation but also their incident illumination and the reflectance of the surfaces to which the photosensors are attached. We use our untethered tag system, called Prakash, to demonstrate methods of adding special effects to captured videos that cannot be accomplished using pure vision techniques that rely on camera images.

UR - http://www.scopus.com/inward/record.url?scp=36949032304&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=36949032304&partnerID=8YFLogxK

U2 - 10.1145/1275808.1276422

DO - 10.1145/1275808.1276422

M3 - Conference contribution

BT - Proceedings of the ACM SIGGRAPH Conference on Computer Graphics

ER -