Bad data processing for water distribution system demand estimation

Doosun Kang, Kevin E Lansey

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Previous studies by the authors have shown that nodal demands in a water distribution system (WDS) can be estimated in real-time using pipe flow data collected by supervisory control and data acquisition (SCADA) system. Estimated demands can be used for optimal operation of system to support pressure and water quality. It is not unusual the data from SCADA systems contain gross errors due to system failure and/or meter malfunctions. The estimator is sensitive to these erroneous measurements and the estimates based on the bad measurements are not reliable for system operation; therefore bad data should be filtered prior to demand estimation. However, system failure and meter malfunctions are random phenomena and hard to identify. This study presents a series of statistical methods to detect bad data, identify their locations, and correct the data values. The proposed methods are based on a linear measurement model that linearly relates state variables (nodal demands) to the field measurements (pipe flow rates). The scheme is applied prior to a demand estimation to eliminate the effects of erroneous data on the demand estimates. The proposed method is applied to a hypothetical simple network using synthetically generated data sets, such as error-free data, Gaussian-noisy data, fire flow data, and noisy data containing one or more contaminated measurements. Application to a simple hypothetical network using synthetically generated data shows that the method can be successfully used as a pre-processing for single and multiple non-interacting bad data for reliable demand estimation.

Original languageEnglish (US)
Title of host publicationWater Distribution Systems Analysis 2010 - Proceedings of the 12th International Conference, WDSA 2010
Pages1248-1255
Number of pages8
DOIs
StatePublished - 2012
Event12th Annual International Conference on Water Distribution Systems Analysis 2010, WDSA 2010 - Tucson, AZ, United States
Duration: Sep 12 2010Sep 15 2010

Other

Other12th Annual International Conference on Water Distribution Systems Analysis 2010, WDSA 2010
CountryUnited States
CityTucson, AZ
Period9/12/109/15/10

Fingerprint

pipe flow
data acquisition
demand
water distribution system
water quality
method
rate
effect

Keywords

  • bad data filter
  • demand estimation
  • SCADA

ASJC Scopus subject areas

  • Water Science and Technology

Cite this

Kang, D., & Lansey, K. E. (2012). Bad data processing for water distribution system demand estimation. In Water Distribution Systems Analysis 2010 - Proceedings of the 12th International Conference, WDSA 2010 (pp. 1248-1255) https://doi.org/10.1061/41203(425)112

Bad data processing for water distribution system demand estimation. / Kang, Doosun; Lansey, Kevin E.

Water Distribution Systems Analysis 2010 - Proceedings of the 12th International Conference, WDSA 2010. 2012. p. 1248-1255.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Kang, D & Lansey, KE 2012, Bad data processing for water distribution system demand estimation. in Water Distribution Systems Analysis 2010 - Proceedings of the 12th International Conference, WDSA 2010. pp. 1248-1255, 12th Annual International Conference on Water Distribution Systems Analysis 2010, WDSA 2010, Tucson, AZ, United States, 9/12/10. https://doi.org/10.1061/41203(425)112
Kang D, Lansey KE. Bad data processing for water distribution system demand estimation. In Water Distribution Systems Analysis 2010 - Proceedings of the 12th International Conference, WDSA 2010. 2012. p. 1248-1255 https://doi.org/10.1061/41203(425)112
Kang, Doosun ; Lansey, Kevin E. / Bad data processing for water distribution system demand estimation. Water Distribution Systems Analysis 2010 - Proceedings of the 12th International Conference, WDSA 2010. 2012. pp. 1248-1255
@inproceedings{3ccee254d8564c09b8b6aac3e59c16e5,
title = "Bad data processing for water distribution system demand estimation",
abstract = "Previous studies by the authors have shown that nodal demands in a water distribution system (WDS) can be estimated in real-time using pipe flow data collected by supervisory control and data acquisition (SCADA) system. Estimated demands can be used for optimal operation of system to support pressure and water quality. It is not unusual the data from SCADA systems contain gross errors due to system failure and/or meter malfunctions. The estimator is sensitive to these erroneous measurements and the estimates based on the bad measurements are not reliable for system operation; therefore bad data should be filtered prior to demand estimation. However, system failure and meter malfunctions are random phenomena and hard to identify. This study presents a series of statistical methods to detect bad data, identify their locations, and correct the data values. The proposed methods are based on a linear measurement model that linearly relates state variables (nodal demands) to the field measurements (pipe flow rates). The scheme is applied prior to a demand estimation to eliminate the effects of erroneous data on the demand estimates. The proposed method is applied to a hypothetical simple network using synthetically generated data sets, such as error-free data, Gaussian-noisy data, fire flow data, and noisy data containing one or more contaminated measurements. Application to a simple hypothetical network using synthetically generated data shows that the method can be successfully used as a pre-processing for single and multiple non-interacting bad data for reliable demand estimation.",
keywords = "bad data filter, demand estimation, SCADA",
author = "Doosun Kang and Lansey, {Kevin E}",
year = "2012",
doi = "10.1061/41203(425)112",
language = "English (US)",
isbn = "9780784412039",
pages = "1248--1255",
booktitle = "Water Distribution Systems Analysis 2010 - Proceedings of the 12th International Conference, WDSA 2010",

}

TY - GEN

T1 - Bad data processing for water distribution system demand estimation

AU - Kang, Doosun

AU - Lansey, Kevin E

PY - 2012

Y1 - 2012

N2 - Previous studies by the authors have shown that nodal demands in a water distribution system (WDS) can be estimated in real-time using pipe flow data collected by supervisory control and data acquisition (SCADA) system. Estimated demands can be used for optimal operation of system to support pressure and water quality. It is not unusual the data from SCADA systems contain gross errors due to system failure and/or meter malfunctions. The estimator is sensitive to these erroneous measurements and the estimates based on the bad measurements are not reliable for system operation; therefore bad data should be filtered prior to demand estimation. However, system failure and meter malfunctions are random phenomena and hard to identify. This study presents a series of statistical methods to detect bad data, identify their locations, and correct the data values. The proposed methods are based on a linear measurement model that linearly relates state variables (nodal demands) to the field measurements (pipe flow rates). The scheme is applied prior to a demand estimation to eliminate the effects of erroneous data on the demand estimates. The proposed method is applied to a hypothetical simple network using synthetically generated data sets, such as error-free data, Gaussian-noisy data, fire flow data, and noisy data containing one or more contaminated measurements. Application to a simple hypothetical network using synthetically generated data shows that the method can be successfully used as a pre-processing for single and multiple non-interacting bad data for reliable demand estimation.

AB - Previous studies by the authors have shown that nodal demands in a water distribution system (WDS) can be estimated in real-time using pipe flow data collected by supervisory control and data acquisition (SCADA) system. Estimated demands can be used for optimal operation of system to support pressure and water quality. It is not unusual the data from SCADA systems contain gross errors due to system failure and/or meter malfunctions. The estimator is sensitive to these erroneous measurements and the estimates based on the bad measurements are not reliable for system operation; therefore bad data should be filtered prior to demand estimation. However, system failure and meter malfunctions are random phenomena and hard to identify. This study presents a series of statistical methods to detect bad data, identify their locations, and correct the data values. The proposed methods are based on a linear measurement model that linearly relates state variables (nodal demands) to the field measurements (pipe flow rates). The scheme is applied prior to a demand estimation to eliminate the effects of erroneous data on the demand estimates. The proposed method is applied to a hypothetical simple network using synthetically generated data sets, such as error-free data, Gaussian-noisy data, fire flow data, and noisy data containing one or more contaminated measurements. Application to a simple hypothetical network using synthetically generated data shows that the method can be successfully used as a pre-processing for single and multiple non-interacting bad data for reliable demand estimation.

KW - bad data filter

KW - demand estimation

KW - SCADA

UR - http://www.scopus.com/inward/record.url?scp=84856160666&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84856160666&partnerID=8YFLogxK

U2 - 10.1061/41203(425)112

DO - 10.1061/41203(425)112

M3 - Conference contribution

SN - 9780784412039

SP - 1248

EP - 1255

BT - Water Distribution Systems Analysis 2010 - Proceedings of the 12th International Conference, WDSA 2010

ER -