How does this not have more views!? Excellent video, EXACTLY what I needed to finish my project at work. This video could have saved me 10 hours of head scratching if I had seen it sooner.
Thank you for the great content. I'm wondering why don't you use early_stopping_rounds during grid search? That way you could set num_trees to a fixed big number (like you did later when building the final model) and don't have to grid search over it. Also, using your approach you probably overfit during grid search (due to the high number of estimators) and only get the best parameters when using all of the 1000, 2000 or 3000 trees. In the final model, due to the fact that you use early_stopping_rounds, a different number of estimators will be used and therefore the optimal hyperparamters from the grid search are probably not the optimal hyperparameters for the final model. What do you think about it?
Hey Alexander, thank you for this good question. You are right, ideally we would want to use something like early_stopping_rounds during grid search. As far as I know, this feature is not available while performing grid search using sklearn. Grid search will check values of all the parameter combinations that have been specified. You are also right in stating that there will be difference in estimators that we get from grid search and from using early_stopping_rounds in the final model. I consider grid search as an initial estimate of what hyperparameters would give better results, but the final model can have slightly different values. Thank you for your interesting question :)
i have a doubt……during cross validation where we choose which model to use i am getting some accuracy but after hyperparameter tuning the accuracy jumps by 2 % Is this normal? This is in XGboost
Nice video! Thank you so much! One pair of doubts, is there a way to download the notebook with outputs from Kaggle? Is it possible to train models like XGBoost with GPU? because the last time I tried there, the debugger suggested that it was only possible with sequential models like neural networks.
Thank you, this was explained really well. I'm working on a scorecard model with over 400 variables, can we use 'from xgboost import plot_importance' to print out the important features post hyper-parameter tuning and training the model and then re-run the model with subset features?
EDA should be done irrespective of the model. Feature selection can also help removing unnecessary complexity in the model. But the benefit for techniques like XGBoost is that it can take in large number of features and give importance to the relevant ones. I would advice doing first iteration with all possible features and then remove features with lower importance, while monitoring model performance metrics.
We are trying to identify which customers will make a specific transaction in the future. These customers will be tagged as 1 in the data. For more details see here www.kaggle.com/competitions/santander-customer-transaction-prediction/overview
How does this not have more views!? Excellent video, EXACTLY what I needed to finish my project at work. This video could have saved me 10 hours of head scratching if I had seen it sooner.
Thanks Matt. I am glad to know that the video helped.
I cannot overstate the fact that this video is really clear and terrific. Absolutely fantastic effort on your part. Thank you very much for doing this
all the advanced terms are simply described. Thanks, Harsh.
Exactly what I needed. Explained very clearly. Thank You.
More videos [like this] that teach optimization of all the parameters in the model, please
Good video sir , Thanks for making videos and educating us
Thank you sir🙏, vidio ini sangat membantu 😊
This video covers a lot of thing in short time
Really nice video and explanation Harsh
Thank you for the great content.
I'm wondering why don't you use early_stopping_rounds during grid search? That way you could set num_trees to a fixed big number (like you did later when building the final model) and don't have to grid search over it. Also, using your approach you probably overfit during grid search (due to the high number of estimators) and only get the best parameters when using all of the 1000, 2000 or 3000 trees.
In the final model, due to the fact that you use early_stopping_rounds, a different number of estimators will be used and therefore the optimal hyperparamters from the grid search are probably not the optimal hyperparameters for the final model. What do you think about it?
Hey Alexander, thank you for this good question. You are right, ideally we would want to use something like early_stopping_rounds during grid search. As far as I know, this feature is not available while performing grid search using sklearn. Grid search will check values of all the parameter combinations that have been specified.
You are also right in stating that there will be difference in estimators that we get from grid search and from using early_stopping_rounds in the final model. I consider grid search as an initial estimate of what hyperparameters would give better results, but the final model can have slightly different values.
Thank you for your interesting question :)
Thanks for the video! Great learning experience.
This is a a very well explained video !
Disliking this video because it’s too good and I don’t want others to know abt it 😂😂
Great video thank you!
Thanks boss
eval_metric throws error, can anyone suggest me the reason?
How can parallelization work in the Xgboost algorithm? Please explain it with an example
I appreciate your effort.
i have a doubt……during cross validation where we choose which model to use i am getting some accuracy but after hyperparameter tuning the accuracy jumps by 2 %
Is this normal?
This is in XGboost
I have a question about the Xgboost algorithm. The question is how parallelization works in the Xgboost algorithm and explain me with an example.
Nice video! Thank you so much!
One pair of doubts, is there a way to download the notebook with outputs from Kaggle? Is it possible to train models like XGBoost with GPU? because the last time I tried there, the debugger suggested that it was only possible with sequential models like neural networks.
Hey man, you doin' a good job! Why u stop making videos?
Thank you very much man. I will start uploading more videos from next month 😀
Really, awesome.
in my project only i get 45% in training and 44 in testing. What do you think i can do to get better accuracy please.
How do you do it for Multiclass classification?
Great content...
Thank you, this was explained really well. I'm working on a scorecard model with over 400 variables, can we use 'from xgboost import plot_importance' to print out the important features post hyper-parameter tuning and training the model and then re-run the model with subset features?
Hi
I'm working on the same
Please help me, with what approach you did
Thanks
The program is too too much time to run 😵
But Thanks to you Sir, for explaining the program and arguments very well.
You can try LightGBM. It may be faster depending on your context. I have a video for it on my channel.
Just wanted to know whether EDA, feature selection is not needed for XGboost ?
EDA should be done irrespective of the model. Feature selection can also help removing unnecessary complexity in the model. But the benefit for techniques like XGBoost is that it can take in large number of features and give importance to the relevant ones. I would advice doing first iteration with all possible features and then remove features with lower importance, while monitoring model performance metrics.
I have a question. What are the two classes here that are being separated.
We are trying to identify which customers will make a specific transaction in the future. These customers will be tagged as 1 in the data. For more details see here www.kaggle.com/competitions/santander-customer-transaction-prediction/overview
where can i get the api of XGboost?
API Reference from Python: xgboost.readthedocs.io/en/stable/python/python_api.html
For other languages, you can see the same website
I think im the stupid one ... the video is in detail but i fail to do ... head scratching moment in spyder :(
where is the data?
You can access data from this link: www.kaggle.com/competitions/santander-customer-transaction-prediction/data
Your justification for learning rate is not right.