On-line augmented reality system by integration of multiple planar markers placed in arbitrary positions and directions

We describe a new on-line AR system using multiple planar markers placed at arbitrary positions and directions. In most marker-based approaches, the markers' 3D positions and directions need to be measured in advance because they need to define a virtual object's coordinate in one fixed 3D coordinate system. However, such measurement is a very difficult and time-consuming task especially when the markers are in a complex arrangement or are placed in a wide space. Thus, we created a new on-line AR system that does not have any information on the geometrical relationship of the markers. In our registration method, we construct projective 3D space using two reference images, in which we can estimate the geometrical relationship of the multiple markers. By defining the virtual object's coordinate in projective space, this coordinate can be represented in one fixed 3D coordinate system for each frame. We performed some experiments to demonstrate the effectiveness of our system. A virtual object was superimposed onto input image sequences captured with a handy camera. Because the multiple markers were placed in a wide area, the virtual objects could easily move around the real world. Moreover, the registration could be carried out stably because the markers face various directions and because some of them can always be recognized from most viewpoints.