Optimizing Control of Wastewater Treatment Plant With Reinforcement Learning: Technical Evaluation of Twin-Delayed Deep Deterministic Policy Gradient Agent

Zuzanna Klawikowska; Michał Grochowski

doi:10.1109/access.2024.3458186

Optimizing Control of Wastewater Treatment Plant With Reinforcement Learning: Technical Evaluation of Twin-Delayed Deep Deterministic Policy Gradient Agent

Abstract

Control of the wastewater treatment processes presents significant challenges due to the fluctuating nature of inflow and wastewater composition, alongside the system’s non-linear dynamics. Traditional control methods struggle to adapt to these variations, leading to an economically suboptimal operation of the process and a violation of norms imposed on the quality of wastewater discharged to the catchment area. This study proposes using a reinforcement learning approach to control a wastewater treatment plant. Specifically, the twin-delayed deep deterministic policy gradient algorithm is employed and comprehensively validated. This algorithm enhances control efficiency, optimizing costs while improving the plant’s wastewater treatment capabilities. The proposed control approach is implemented and evaluated using Benchmark Simulation Model No. 1. The evaluation is based on four representative weather scenarios and examines sixteen different metrics. In the proposed case study, an agent controls oxygen transfer coefficients in three oxygen tanks of a wastewater treatment plant. Two distinct reward functions are employed to guide the reinforcement learning agent. The first one focuses on maintaining aerobic conditions and minimizing total nitrogen. The second function retains these objectives and also aims to minimize the aeration energy. The experimental results demonstrate that the proposed control strategy significantly enhances aeration energy efficiency and reduces the overall cost index, indicating a more cost-effective operation. However, future work is required to improve further the performance of reinforcement learning that optimizes the wastewater treatment process.

Citations

0

CrossRef
0

Web of Science
0

Scopus

Authors (2)

Cite as

Full text

full text is not available in portal

full content of the article see on external site open in new tab

Keywords

Details

Category:

Articles

Type:

artykuły w czasopismach

Published in:

IEEE Access no. 12, pages 130318 - 130333,
ISSN: 2169-3536

Language:

English

Publication year:

2024

Bibliographic description:

Klawikowska Z., Grochowski M.: Optimizing Control of Wastewater Treatment Plant With Reinforcement Learning: Technical Evaluation of Twin-Delayed Deep Deterministic Policy Gradient Agent// IEEE Access -, (2024), s.130318-130333

DOI: