ReallifeEngine: A Mixed Reality-Based Visual Programming System for SmartHomes

The conveniences experienced by society have tremendously improved with the development of the Internet of Things (IoT). Among the affordances stemming from this innovation is an IoT concept called the SmartHome, which is already spreading even in general households. Despite this proliferation, however, ordinary users experience difficulty in performing the complex control and automation of IoT devices, thereby impeding their full exploitation of IoT benefits. These problems highlight the need for a system that enables general users to easily manipulate IoT devices. Correspondingly, this study constructed a visual programming system that facilitates IoT device operation. The system, which was developed on the basis of data obtained from various sensors in a SmartHome, employs mixed reality(MR) in enhancing the visualization of various data, eases the understanding of the positional relationship among devices, and smoothens the checking of execution results. We conducted an evaluation experiment wherein eight users were asked to test the proposed system, and we verified its usefulness on the basis of the time elapsed until the participants completed the programming of diverse IoT devices and a questionnaire intended to derive their subjective assessments. The result indicates that the proposed system makes it easy to understand the correspondence between the real world device and the node in the MR environment, and the connection between the sensors and the home appliances. On the other hand, it is negatively evaluated for operability.