Bilateral control can realize haptic feedback required for securing safety and diversity in teleoperation by robots. Since conventional approaches are based on a servo controller which has high-performance current control or torque control, there is a barrier for advancing into society. The objective of this paper is to propose a design method of the bilateral controller using velocity control driver to achieve haptic feedback. The velocity control driver such as an inverter is widely used in the industrial fields. In the proposed method, a reaction force observer modified for velocity control driver is used to estimate environmental reaction force. The control system is constructed by applying the concept of hybrid control which controls velocity and force independently, and the concept of modal space which separates motions into functions is also applied. Furthermore, the proposed bilateral controller is designed based on the ideal conditions of operationality and reproducibility. The validity of the proposal is verified through a simulation and an experiment. The obtained results show that the proposed controller successfully achieves the teleoperation with haptic feedback using velocity control driver.