Adaptive control method of HVAC for uniformizing comfort at Japanese residential living rooms using deep reinforcement learning

There is a great demand for high-quality indoor thermal environments while achieving comfort and saving the energy for heating, ventilation, and air conditioning (HVAC) devices offer a tradeoff. The room shapes, partitioning, and the location of diffusers cause uneven comfort and degrade thermal satisfaction. Therefore, an HVAC control method that provides uniform thermal comfort while saving energy is required. In such cases, providing uniform comfort in a room can be a low-cost and energy-efficient solution. We propose a deep reinforcement learning (DRL)-based HVAC control method to uniformize comfort in a room. The proposed method considers the use of sensors, which are already deployed in existing HVAC devices and is adaptive to rooms with different conditions. The proposed method controls the HVAC device based on the prediction of the room's thermal response using wall heatmaps of the target room. The heatmaps are obtained by using a simple measuring system of an infrared thermopile array sensor. For evaluating the proposed method's adaptivity to rooms with different conditions, a computer flud dynamics (CFD)-based simulation was designed. The coldest period throughout a year was chosen as the evaluation period; uniformizing comfort is the most difficult when the temperature difference between outside and inside a room and the validity of the proposed method for the other periods is implied. The usefulness of the proposed method for the chosen period was shown for the selected period; the proposed method achieved an improvement of 33.9% in thermal comfort performance while on average increasing energy consumption by 6.3%, against that of comparable approaches. Moreover, the comfort performance during learning is confirmed to be similar to those of the existing method.