Input and display of hand drawn pattern onto planar board of arbitrary position and pose utilizing a projector and two cameras

In this paper, we propose a system for inputting hand drawn pattern and displaying it on a hand-held planar rectangle board by utilizing two cameras and a projector which are installed above the user, In this system, the cameras and the projector are related by projective geometry. The cameras capture the user's hand-drawing motion. From the captured images, the tip of the user's pen and the corners of the board are tracked for detecting the status of pen's up/down and displaying the drawn pattern at a fixed position on the surface of the board. For demonstrating the validity of the proposed system, we show that the user can draw pattern on the board with arbitrary pose and position, while the pattern is simultaneously displayed on the board in real-time.