In this paper, we aimed to extract time delay in stress fluctuation during excavation work to realize a control system of hydraulic excavators (KANSEI feedback control system). In recent years, there is a growing demand to remove stress caused by harsh working environments, because they may cause life-threatening diseases including depression. We focused on construction sites as examples of harsh working environments. Attempts have been made to reduce noise and vibration of hydraulic excavators to improve the environment surrounding workers. These attempts do not consider operability however, which is also a considerable cause of stress. We attempted to improve the operability of hydraulic excavators by realizing automatic control system using stress values (KANSEI feedback control system). In order to realize this system, there are two phases: Finding correlation between control output and stress (phase 1), and finding correlation between control input and stress (phase 2). In this research, we aimed to achieve phase 1, and extracted the time delay in stress fluctuation of operators with different skill levels using different types of hydraulic excavators. As a result, it was confirmed that the time delay tended to increase with working time.