Control of time delay system has been researched for a long time. In time delay system, feedback including time delay sometimes makes transient response of a system worse. In the worst case, the system becomes unstable. As time delay compensation method, various methods have been proposed, for example, smith predictor, communication disturbance observer, and so on. In particular, communication disturbance observer is superior than smith predictor in terms of no using a time delay model. However, if an error in plant model exists or disturbance is applied to the system, steady-state error occurs. As disturbance rejection method, communication disturbance observer with an disturbance observer and an integrator in outer loop was proposed in conventional research. But using this method, time delay compensation and disturbance rejection can not be designed independently. Therefore, in this paper, an integrator is not used and two degrees of freedom control is conducted by communication disturbance observer and disturbance observer. For eliminating disturbance, a novel filter design method is proposed. Using proposed filter, the effect of disturbance is suppressed. Finally, the validity of the proposed method is verified by simulation results.