Recently, haptic information is studied as the third multimedia information. Real-world haptics is getting attention as not only the principle for feedback of real-world haptic information in teleoperation but also the key technology for future human support. In order to control haptic information, the bilateral control with a master-slave system is used widely. When the plural systems are operated, it is desirable that the system carries out a complementary work autonomously in order to realize the desired task of the operator. Therefore, in this paper, the control technique that considers work and control structure hierarchically is proposed. By this method, the operator can concentrate on a desired task while a system realizes adaptation to the environment autonomously. In this paper, one-to-many bilateral control is realized. The bilateral remote control could be realized while the slave system adapts to the environment autonomously. Moreover, the operator can operate the system without minding the change of the environmental configuration and posture.