Deep reinforcement learning-based prosumer aggregation bidding strategy in a hierarchical local electricity market

Haoyang Zhang; J.K. (Koen) Kok; N.G. Paterakis

doi:10.1109/EEE-AM58328.2023.10395533

Deep reinforcement learning-based prosumer aggregation bidding strategy in a hierarchical local electricity market

Haoyang Zhang, J.K. (Koen) Kok, N.G. Paterakis

Onderzoeksoutput: Hoofdstuk in Boek/Rapport/Congresprocedure › Conferentiebijdrage › Academic › peer review

1 Citaat (Scopus)

76 Downloads (Pure)

Samenvatting

This paper investigates the application of deep reinforcement learning (DRL) algorithm for the decision-support of a prosumer aggregation in a hierarchical local electricity market (LEM) comprising a peer-to-peer (P2P) market and a corrective market. The agent first submits bids/asks to the P2P market where prosumer aggregations are able to trade electricity directly with each other. After that, the agent participates in the corrective market, where the market operator formulates the corrective market as an AC optimal power flow (OPF) problem to ensure the system is operated within its operational limits. A DRL algorithm, namely Twin Delayed Deep Deterministic Policy Gradient (TD3), is used to find the strategic bidding strategy. The algorithm is tested on a real medium-voltage distribution grid to evaluate the effectiveness of the strategic bidding method. The result of the case study demonstrates that the agent can derive trading strategies to obtain high profits based on the TD3 algorithm.

Originele taal-2	Engels
Titel	2023 Asia Meeting on Environment and Electrical Engineering, EEE-AM 2023
Redacteuren	Zbigniew Leonowicz, Erika Stracqualursi
Uitgeverij	Institute of Electrical and Electronics Engineers
Aantal pagina's	6
ISBN van elektronische versie	979-8-3503-8106-1
DOI's	https://doi.org/10.1109/EEE-AM58328.2023.10395533
Status	Gepubliceerd - 25 jan. 2024
Evenement	1st Asia Meeting on Environment and Electrical Engineering - Hanoi, Vietnam Duur: 13 nov. 2023 → 15 nov. 2023

Congres

Congres	1st Asia Meeting on Environment and Electrical Engineering
Verkorte titel	EEE-AM 2023
Land/Regio	Vietnam
Stad	Hanoi
Periode	13/11/23 → 15/11/23

Financiering

This publication is part of the research program ‘MEGAMIND – Enabling distributed operation of energy infrastructures through Measuring, Gathering, Mining and Integrating grid-edge Data’, (partly) financed by the Dutch Research Council (NWO), through the Perspectief funding instrument under number P19-25.

Financiers	Financiernummer
Nederlandse Organisatie voor Wetenschappelijk Onderzoek	P19-25

Duurzame ontwikkelingsdoelstellingen van de VN

Deze output draagt bij aan de volgende duurzame ontwikkelingsdoelstelling(en)

Toegang tot document

10.1109/EEE-AM58328.2023.10395533

pdfDefinitieve gepubliceerde versie, 710 KBLicentie: TAVERNE

Andere bestanden en links

Link naar publicatie in Scopus

Citeer dit

Zhang, H., Kok, J. K., & Paterakis, N. G. (2024). Deep reinforcement learning-based prosumer aggregation bidding strategy in a hierarchical local electricity market. In Z. Leonowicz, & E. Stracqualursi (editors), 2023 Asia Meeting on Environment and Electrical Engineering, EEE-AM 2023 Artikel 10395533 Institute of Electrical and Electronics Engineers. https://doi.org/10.1109/EEE-AM58328.2023.10395533

@inproceedings{888f916dfb714a4ab58147f35dce0009,

title = "Deep reinforcement learning-based prosumer aggregation bidding strategy in a hierarchical local electricity market",

abstract = "This paper investigates the application of deep reinforcement learning (DRL) algorithm for the decision-support of a prosumer aggregation in a hierarchical local electricity market (LEM) comprising a peer-to-peer (P2P) market and a corrective market. The agent first submits bids/asks to the P2P market where prosumer aggregations are able to trade electricity directly with each other. After that, the agent participates in the corrective market, where the market operator formulates the corrective market as an AC optimal power flow (OPF) problem to ensure the system is operated within its operational limits. A DRL algorithm, namely Twin Delayed Deep Deterministic Policy Gradient (TD3), is used to find the strategic bidding strategy. The algorithm is tested on a real medium-voltage distribution grid to evaluate the effectiveness of the strategic bidding method. The result of the case study demonstrates that the agent can derive trading strategies to obtain high profits based on the TD3 algorithm.",

keywords = "AC OPF, Deep reinforcement learning, Peer-to-peer market, Strategic bidding",

author = "Haoyang Zhang and Kok, \{J.K. (Koen)\} and N.G. Paterakis",

year = "2024",

month = jan,

day = "25",

doi = "10.1109/EEE-AM58328.2023.10395533",

language = "English",

editor = "Zbigniew Leonowicz and Erika Stracqualursi",

booktitle = "2023 Asia Meeting on Environment and Electrical Engineering, EEE-AM 2023",

publisher = "Institute of Electrical and Electronics Engineers",

address = "United States",

note = "1st Asia Meeting on Environment and Electrical Engineering, EEE-AM 2023 ; Conference date: 13-11-2023 Through 15-11-2023",

}

Zhang, H , Kok, JK & Paterakis, NG 2024, Deep reinforcement learning-based prosumer aggregation bidding strategy in a hierarchical local electricity market. in Z Leonowicz & E Stracqualursi (uitgave), 2023 Asia Meeting on Environment and Electrical Engineering, EEE-AM 2023., 10395533, Institute of Electrical and Electronics Engineers, 1st Asia Meeting on Environment and Electrical Engineering, Hanoi, Vietnam, 13/11/23. https://doi.org/10.1109/EEE-AM58328.2023.10395533

Deep reinforcement learning-based prosumer aggregation bidding strategy in a hierarchical local electricity market. / Zhang, Haoyang ; Kok, J.K. (Koen); Paterakis, N.G.
2023 Asia Meeting on Environment and Electrical Engineering, EEE-AM 2023. uitgave / Zbigniew Leonowicz; Erika Stracqualursi. Institute of Electrical and Electronics Engineers, 2024. 10395533.

Onderzoeksoutput: Hoofdstuk in Boek/Rapport/Congresprocedure › Conferentiebijdrage › Academic › peer review

TY - GEN

T1 - Deep reinforcement learning-based prosumer aggregation bidding strategy in a hierarchical local electricity market

AU - Zhang, Haoyang

AU - Kok, J.K. (Koen)

AU - Paterakis, N.G.

PY - 2024/1/25

Y1 - 2024/1/25

N2 - This paper investigates the application of deep reinforcement learning (DRL) algorithm for the decision-support of a prosumer aggregation in a hierarchical local electricity market (LEM) comprising a peer-to-peer (P2P) market and a corrective market. The agent first submits bids/asks to the P2P market where prosumer aggregations are able to trade electricity directly with each other. After that, the agent participates in the corrective market, where the market operator formulates the corrective market as an AC optimal power flow (OPF) problem to ensure the system is operated within its operational limits. A DRL algorithm, namely Twin Delayed Deep Deterministic Policy Gradient (TD3), is used to find the strategic bidding strategy. The algorithm is tested on a real medium-voltage distribution grid to evaluate the effectiveness of the strategic bidding method. The result of the case study demonstrates that the agent can derive trading strategies to obtain high profits based on the TD3 algorithm.

AB - This paper investigates the application of deep reinforcement learning (DRL) algorithm for the decision-support of a prosumer aggregation in a hierarchical local electricity market (LEM) comprising a peer-to-peer (P2P) market and a corrective market. The agent first submits bids/asks to the P2P market where prosumer aggregations are able to trade electricity directly with each other. After that, the agent participates in the corrective market, where the market operator formulates the corrective market as an AC optimal power flow (OPF) problem to ensure the system is operated within its operational limits. A DRL algorithm, namely Twin Delayed Deep Deterministic Policy Gradient (TD3), is used to find the strategic bidding strategy. The algorithm is tested on a real medium-voltage distribution grid to evaluate the effectiveness of the strategic bidding method. The result of the case study demonstrates that the agent can derive trading strategies to obtain high profits based on the TD3 algorithm.

KW - AC OPF

KW - Deep reinforcement learning

KW - Peer-to-peer market

KW - Strategic bidding

UR - http://www.scopus.com/inward/record.url?scp=85184849077&partnerID=8YFLogxK

U2 - 10.1109/EEE-AM58328.2023.10395533

DO - 10.1109/EEE-AM58328.2023.10395533

M3 - Conference contribution

BT - 2023 Asia Meeting on Environment and Electrical Engineering, EEE-AM 2023

A2 - Leonowicz, Zbigniew

A2 - Stracqualursi, Erika

PB - Institute of Electrical and Electronics Engineers

T2 - 1st Asia Meeting on Environment and Electrical Engineering

Y2 - 13 November 2023 through 15 November 2023

ER -

Zhang H , Kok JK , Paterakis NG. Deep reinforcement learning-based prosumer aggregation bidding strategy in a hierarchical local electricity market. In Leonowicz Z, Stracqualursi E, redacteurs, 2023 Asia Meeting on Environment and Electrical Engineering, EEE-AM 2023. Institute of Electrical and Electronics Engineers. 2024. 10395533 doi: 10.1109/EEE-AM58328.2023.10395533

Deep reinforcement learning-based prosumer aggregation bidding strategy in a hierarchical local electricity market

Samenvatting

Congres

Financiering

Duurzame ontwikkelingsdoelstellingen van de VN

Toegang tot document

Andere bestanden en links

Vingerafdruk

Citeer dit