Distributed Deep Reinforcement Learning for Renewable Energy Accommodation Assessment with Communication Uncertainty in Internet of Energy

Nowadays, microgrids (MG) have attracted much attention, as a key technology of the Internet of Energy (IoE). A great deal of research have shown that the hierarchical microgrid is a more novel structure of IoE. Although the hierarchical microgrid model solves the problem of weak power scheduling capability across microgrids, it suffers from severe communications uncertainty, which can lead to communication delay and fluctuation. To obtain the accurate result of the renewable energy accommodation assessment capacity, a hierarchical microgrid model considering communication uncertainty is proposed in this article. The solution to solve the problem of the assessment renewable energy accommodation capacity for hierarchical MG is a hybrid control based on distribution deep reinforcement learning. The temporal difference (TD) generation adversarial network (TD-GAN) is proposed as a value-based method. Compared with the policy-based method, it can better solve the distributed problem in hybrid control with a generation adversarial network (GAN). Moreover, the challenge that the method cannot handle a continuous action space is solved by using a normalized advantage function (NAF). The method similar with the TD error method is employed to train the GAN network. Simulation results using real power grid data demonstrate the effectiveness and accuracy of the proposed method.