Optimizing Control of Wastewater Treatment Plant With Reinforcement Learning: Technical Evaluation of Twin-Delayed Deep Deterministic Policy Gradient Agent

Zuzanna Klawikowska; Michał Grochowski

doi:10.1109/access.2024.3458186

Optimizing Control of Wastewater Treatment Plant With Reinforcement Learning: Technical Evaluation of Twin-Delayed Deep Deterministic Policy Gradient Agent

Abstrakt

Control of the wastewater treatment processes presents significant challenges due to the fluctuating nature of inflow and wastewater composition, alongside the system’s non-linear dynamics. Traditional control methods struggle to adapt to these variations, leading to an economically suboptimal operation of the process and a violation of norms imposed on the quality of wastewater discharged to the catchment area. This study proposes using a reinforcement learning approach to control a wastewater treatment plant. Specifically, the twin-delayed deep deterministic policy gradient algorithm is employed and comprehensively validated. This algorithm enhances control efficiency, optimizing costs while improving the plant’s wastewater treatment capabilities. The proposed control approach is implemented and evaluated using Benchmark Simulation Model No. 1. The evaluation is based on four representative weather scenarios and examines sixteen different metrics. In the proposed case study, an agent controls oxygen transfer coefficients in three oxygen tanks of a wastewater treatment plant. Two distinct reward functions are employed to guide the reinforcement learning agent. The first one focuses on maintaining aerobic conditions and minimizing total nitrogen. The second function retains these objectives and also aims to minimize the aeration energy. The experimental results demonstrate that the proposed control strategy significantly enhances aeration energy efficiency and reduces the overall cost index, indicating a more cost-effective operation. However, future work is required to improve further the performance of reinforcement learning that optimizes the wastewater treatment process.

Cytowania

0

CrossRef
0

Web of Science
0

Scopus

Autorzy (2)

Cytuj jako

Pełna treść

pełna treść publikacji nie jest dostępna w portalu

pełna treść artykułu zobacz w serwisie zewnętrznym otwiera się w nowej karcie

Słowa kluczowe

Informacje szczegółowe

Kategoria:

Publikacja w czasopiśmie

Typ:

artykuły w czasopismach

Opublikowano w:

IEEE Access nr 12, strony 130318 - 130333,
ISSN: 2169-3536

Język:

angielski

Rok wydania:

2024

Opis bibliograficzny:

Klawikowska Z., Grochowski M.: Optimizing Control of Wastewater Treatment Plant With Reinforcement Learning: Technical Evaluation of Twin-Delayed Deep Deterministic Policy Gradient Agent// IEEE Access -, (2024), s.130318-130333

DOI: