XGBoost
- XGBoost (Extreme Gradient Boosting) is a specific machine learning algorithm used for supervised learning tasks, such as classification and regression.
- It is based on the gradient boosting framework, which combines multiple weak models (typically decision trees) to create a strong predictive model.
- XGBoost is known for its efficiency, scalability, and high performance, making it a popular choice in machine learning competitions and real-world applications.
XGBoost Parameters
| Param Name | Description | Default Values | Possible Values |
|---|---|---|---|
| MAX_DEPTH | Maximum depth of a tree. Controls overfitting. | 6 | |
| ETA | Step size shrinkage used in update to prevent overfitting. |
0.3 | |
| NROUNDS | Number of boosting rounds (iterations). | 100 | |
| BOOSTER | Type of boosting model to use (e.g., 'gbtree', 'gblinear', 'dart'). |
gbtree | gbtree, gblinear, dart |
| OBJECTIVE | Objective function to optimize (e.g., regression, classification). |
reg:squarederror | regression, classification |
| SUBSAMPLE | Fraction of training data used for each boosting round to prevent overfitting. |
1 | |
| COLSAMPLE_BYTREE | Fraction of features to be randomly sampled for each tree. |
1 | |
| GAMMA | Minimum loss reduction required to make a further partition on a leaf node. |
0 | |
| MIN_CHILD_WEIGHT | Minimum sum of instance weight (hessian) in a child. Controls overfitting. |
1 | |
| LAMBDA | L2 regularization term on weights. Helps prevent overfitting. |
1 | |
| ALPHA | L1 regularization term on weights. Helps prevent overfitting. |
0 | |
| EVAL_METRIC | Evaluation metric for validation set (e.g., 'rmse', 'logloss'). | rmse | rmse, logloss |
| NTHREAD | Number of threads to use for training. | 4 |