Diminished reality using real-time region extraction with depth cameras

In contrast with augmented reality (AR), diminished reality (DR) is a technique for removing or concealing real objects from the real world by overlaying a background onto an area with obstacles in captured images. We propose a real time DR system that is an extension of our previous DR system used in baseball games. Our proposed system consists of three colors cameras and one depth camera. Thanks to the depth camera, the area of obstacles can be detected in real time on the basis of the distance. In an experiment, all cameras were connected to a PC, and then a catcher's viewpoint images were generated in real time by removing the catcher and umpire (obstacles) and visualizing the pitcher pitching the ball (background) via planar projection with two color cameras. The transparency of the obstacles can be changed according to the user's preference.