Effect of Outliers on Performance of Machine Learning Models: An Empirical Analysis

Authors

  • Hussein Bakiri Institute of Finance Management
  • Hadija Mbembati University of Dar es Salaam

Abstract

Machine learning (ML) is a key component of artificial intelligence, capable of solving complex problems by mimicking natural behaviors. Existing studies on the outlier effect in machine learning typically examine fewer than 10 outliers per dataset, limiting analysis of how outlier trends influence model performance. This study addresses this gap through an empirical analysis using 200 outlier instances. The data were drawn from 105,192 electricity consumption records, of which 61,783 were identified as outliers. Four ML models were applied: Support Vector Regression (SVR), Artificial Neural Networks (ANN), Decision Trees (DT), and Random Forests (RF). Results indicate that the relationship between outliers and training accuracy is linear for SVR and RF. In contrast, ANN and DT exhibit nonlinear trends, especially when the outlier percentage falls below a critical threshold of 13.978%. In terms of performance, DT, RF, and SVR show lower sensitivity to outliers compared to ANN. These findings highlight the importance of careful model selection when handling datasets with significant outlier presence.

☍ View Article Analytics

Author Biographies

Hussein Bakiri, Institute of Finance Management

Department of Computer Science

Hadija Mbembati , University of Dar es Salaam

Department of Electronics and Telecommunications Engineering

Downloads

Published

2025-12-30

Issue

Section

Articles