Texture overlay onto non-rigid surface using commodity depth camera

We present a method for overlaying a texture onto a non-rigid surface using a commodity depth camera. The depth cameras are able to capture 3-D data of a surface in real-time, and have several advantages compared with methods using only standard color cameras. However, it is not easy to register a 3-D deformable mesh to a point cloud of the non-rigid surface while keeping its geometrical topology. In order to solve this problem, our method starts by learning many representative meshes to generate surface deformation models. Then, while capturing 3-D data, we register a feasible 3-D mesh to the target surface and overlay a template texture onto the registered mesh. Even if the depth data are noisy or sparse, the learning-based method provides us with a smooth surface mesh. In addition, our method can be applied to real-time applications. In our experiments, we show some augmented reality results of texture overlay onto a non-textured T-shirt.