Paper published in a book (Scientific congresses, symposiums and conference proceedings)
On the Reduction of Biases in Big Data Sets for the Detection of Irregular Power Usage
Glauner, Patrick; State, Radu; Valtchev, Petko et al.
2018In Proceedings 13th International FLINS Conference on Data Science and Knowledge Engineering for Sensing Decision Support (FLINS 2018)
Peer reviewed
 

Files


Full Text
On the Reduction of Biases in Big Data Sets for the Detection of Irregular Power Usage.pdf
Author postprint (281.77 kB)
Download

All documents in ORBilu are protected by a user license.

Send to



Details



Keywords :
Bias; Class imbalance; Covariate shift; Non-technical losses
Abstract :
[en] In machine learning, a bias occurs whenever training sets are not representative for the test data, which results in unreliable models. The most common biases in data are arguably class imbalance and covariate shift. In this work, we aim to shed light on this topic in order to increase the overall attention to this issue in the field of machine learning. We propose a scalable novel framework for reducing multiple biases in high-dimensional data sets in order to train more reliable predictors. We apply our methodology to the detection of irregular power usage from real, noisy industrial data. In emerging markets, irregular power usage, and electricity theft in particular, may range up to 40% of the total electricity distributed. Biased data sets are of particular issue in this domain. We show that reducing these biases increases the accuracy of the trained predictors. Our models have the potential to generate significant economic value in a real world application, as they are being deployed in a commercial software for the detection of irregular power usage.
Disciplines :
Computer science
Author, co-author :
Glauner, Patrick ;  University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT)
State, Radu  ;  University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT)
Valtchev, Petko;  University of Quebec in Montreal
Duarte, Diogo;  CHOICE Technologies Holding Sàrl
External co-authors :
yes
Language :
English
Title :
On the Reduction of Biases in Big Data Sets for the Detection of Irregular Power Usage
Publication date :
2018
Event name :
13th International FLINS Conference on Data Science and Knowledge Engineering for Sensing Decision Support (FLINS 2018)
Event date :
from 21-08-2018 to 24-08-2018
Audience :
International
Main work title :
Proceedings 13th International FLINS Conference on Data Science and Knowledge Engineering for Sensing Decision Support (FLINS 2018)
Peer reviewed :
Peer reviewed
Focus Area :
Computational Sciences
Available on ORBilu :
since 05 April 2018

Statistics


Number of views
84 (11 by Unilu)
Number of downloads
112 (1 by Unilu)

Bibliography


Similar publications



Contact ORBilu