Effect of Outliers on Performance of Machine Learning Models: An Empirical Analysis
Abstract
Machine learning (ML) is a key component of artificial intelligence, capable of solving complex problems by mimicking natural behaviors. Existing studies on the outlier effect in machine learning typically examine fewer than 10 outliers per dataset, limiting analysis of how outlier trends influence model performance. This study addresses this gap through an empirical analysis using 200 outlier instances. The data were drawn from 105,192 electricity consumption records, of which 61,783 were identified as outliers. Four ML models were applied: Support Vector Regression (SVR), Artificial Neural Networks (ANN), Decision Trees (DT), and Random Forests (RF). Results indicate that the relationship between outliers and training accuracy is linear for SVR and RF. In contrast, ANN and DT exhibit nonlinear trends, especially when the outlier percentage falls below a critical threshold of 13.978%. In terms of performance, DT, RF, and SVR show lower sensitivity to outliers compared to ANN. These findings highlight the importance of careful model selection when handling datasets with significant outlier presence.
