Hard Drive Failure Prediction for Large Scale Storage System
Data centers use large numbers of hard drives as data storage devices and it is an increasing challenge to maintain the reliability of the storage system as the number of the hard drives increases exponentially. Manual monitoring does not seem to be efficient for large scale storage systems. Typically, the distributions of healthy hard drives and failed hard drives are highly imbalance. In addition, the size of the training data is large for large scale storage systems. The existence of such challenges makes the hard drive failure prediction problem interesting. In this thesis, several classification models are applied to the hard drive S.M.A.R.T. data from 34,970 hard drives for failure prediction, and the results are compared. Based on the analysis, XGBoost provides the best overall prediction result and it is able to process the data efficiently.