Mitigating Multicollinearity and Overfitting in Car Price Prediction: A comparative Analysis of Regularization and Ensemble Learning Techniques

Abstract

This study examines the effects of multicollinearity and overfitting on car price prediction using linear regression models and evaluates the effectiveness of regularization techniques in addressing these challenges. Multicollinearity, characterized by high correlations among predictor variables, inflates the variance of estimated regression coefficients and leads to
unstable and unreliable predictions. Overfitting arises when a model captures random noise in the training data rather than the true underlying relationships, thereby reducing its generalization capability. To mitigate these issues, ridge regression and lasso regression are employed. Ridge regression introduces an L2 penalty that shrinks coefficient estimates, reducing variance and improving model stability. Lasso regression incorporates an L1 penalty, which not only regularizes the model but also performs variable selection by shrinking some coefficients exactly to zero. A comparative analysis conducted on a car
price prediction dataset demonstrates that both regularization methods yield more robust, stable, and interpretable models compared to ordinary least squares regression. Furthermore, the study extends the analysis to ensemble learning approaches, including Random Forest and XGBoost, to evaluate their performance in the presence of multicollinearity and overfitting. The findings provide insights into the relative strengths of regularized and ensemble methods for improving predictive accuracy and model reliability.