TY - JOUR
T1 - Distributed Deep Reinforcement Learning for Renewable Energy Accommodation Assessment with Communication Uncertainty in Internet of Energy
AU - Fang, Dawei
AU - Guan, Xin
AU - Peng, Yu
AU - Chen, Hongyang
AU - Ohtsuki, Tomoaki
AU - Han, Zhu
N1 - Funding Information:
Manuscript received April 19, 2020; revised August 27, 2020 and October 20, 2020; accepted December 15, 2020. Date of publication December 22, 2020; date of current version May 7, 2021. This work was supported in part by the Science and Technology Projects of State Grid Corporation of China under Grant SGHL0000DKJS2002008 and in part by the U.S. Multidisciplinary University Research Initiative under Grant 18RT0073, Grant NSF EARS-1839818, Grant CNS1717454, Grant CNS-1731424, and Grant CNS-1702850. (Corresponding author: Xin Guan.) Dawei Fang and Xin Guan are with the School of Data Science and Technology, Heilongjiang University, Harbin 150080, China (e-mail: 2181782@s.hlju.edu.cn; guanxin.hlju@gmail.com).
Publisher Copyright:
© 2014 IEEE.
PY - 2021/5/15
Y1 - 2021/5/15
N2 - Nowadays, microgrids (MG) have attracted much attention, as a key technology of the Internet of Energy (IoE). A great deal of research have shown that the hierarchical microgrid is a more novel structure of IoE. Although the hierarchical microgrid model solves the problem of weak power scheduling capability across microgrids, it suffers from severe communications uncertainty, which can lead to communication delay and fluctuation. To obtain the accurate result of the renewable energy accommodation assessment capacity, a hierarchical microgrid model considering communication uncertainty is proposed in this article. The solution to solve the problem of the assessment renewable energy accommodation capacity for hierarchical MG is a hybrid control based on distribution deep reinforcement learning. The temporal difference (TD) generation adversarial network (TD-GAN) is proposed as a value-based method. Compared with the policy-based method, it can better solve the distributed problem in hybrid control with a generation adversarial network (GAN). Moreover, the challenge that the method cannot handle a continuous action space is solved by using a normalized advantage function (NAF). The method similar with the TD error method is employed to train the GAN network. Simulation results using real power grid data demonstrate the effectiveness and accuracy of the proposed method.
AB - Nowadays, microgrids (MG) have attracted much attention, as a key technology of the Internet of Energy (IoE). A great deal of research have shown that the hierarchical microgrid is a more novel structure of IoE. Although the hierarchical microgrid model solves the problem of weak power scheduling capability across microgrids, it suffers from severe communications uncertainty, which can lead to communication delay and fluctuation. To obtain the accurate result of the renewable energy accommodation assessment capacity, a hierarchical microgrid model considering communication uncertainty is proposed in this article. The solution to solve the problem of the assessment renewable energy accommodation capacity for hierarchical MG is a hybrid control based on distribution deep reinforcement learning. The temporal difference (TD) generation adversarial network (TD-GAN) is proposed as a value-based method. Compared with the policy-based method, it can better solve the distributed problem in hybrid control with a generation adversarial network (GAN). Moreover, the challenge that the method cannot handle a continuous action space is solved by using a normalized advantage function (NAF). The method similar with the TD error method is employed to train the GAN network. Simulation results using real power grid data demonstrate the effectiveness and accuracy of the proposed method.
KW - Communication uncertainty
KW - Internet of Energy (IoE)
KW - deep reinforcement learning
KW - generation adversarial network (GAN)
KW - hierarchical microgrid
KW - normalized advantage functions (NAFs)
UR - http://www.scopus.com/inward/record.url?scp=85098774713&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85098774713&partnerID=8YFLogxK
U2 - 10.1109/JIOT.2020.3046622
DO - 10.1109/JIOT.2020.3046622
M3 - Article
AN - SCOPUS:85098774713
VL - 8
SP - 8557
EP - 8569
JO - IEEE Internet of Things Journal
JF - IEEE Internet of Things Journal
SN - 2327-4662
IS - 10
M1 - 9302578
ER -