Hi Jeff. I've recently subscribed and I honestly have to say you have the most comprehensive and easy to understand guides out there. Not to mention the fact that whenever there is an update to something, you make a new video explaining how to work with it. I tried getting in to machine learning just over a year ago and nobody at the time was able to actually explain anything apart from "download this, download that, if it doesn't work oh well" and would just go through the official tutorials without actually explaining how to do anything on your own. Your channel alone has given me the motivation to get started again and thank you so much for doing what you're doing!
Amazing video Jeff ! The only thing you didn't tell us is if you then drop the source features to avoid collinearity or you just leave them along with the new features you created .... Or you perform PCA, VIF or Lasso after it to chose what to do?.... I loved the video concise and super useful!
@@jameswilliamson1726 I will often standardize/norm after applying these techniques. The techniques I use here are really to capture the interaction between underlying features. Then standardization/normlization on top solves range concerns.
Yes for sure, so you must keep that in mind when evaluating feature importance. Generally, I leave the existing features in and let the model account for that (though some model types perform better with correlating fields removed).
Hi Jeff, what concepts should I look into to understand "Weighting" better? For instance at 9:41, you mention that if one values food more they might square it. Someone might cube it, someone might multiply it or add a coefficient of 2 or 5. These are all subjective. For weighting when it comes to features in the stock market or econometrics (my specific application), one might have a feature that is GDP or inflation. I know for a fact that change in GDP (slope) and change in the change in GDP (slope of slope i.e., acceleration) are pretty important. My first problem, is that I found these two (change in GDP and GDP acceleration) simply through guess and check, and research papers. Is there a better method to this? Or should I focus on automating 'guess and check'? Secondly, sometimes the GDP features or inflation related features vary in importance to participants in the stock market. Perhaps right now (as of Oct 2022) investors might place more emphasis on inflation related features and so I might multiply inflation features by coefficient of 2 or square it. How would one deal with dynamic weighting? Or a simpler problem might be, how do you objectively select for weighting? EDIT: I have come up with an idea, to add a coefficient to GDP or inflation based on social media mentions (sentiment), for instance. Thoughts on this and weighting in general? Thanks so much! Love the video by the way!
Thanks! Why would you e.g. square variables to make them more dominant in the model? Wouldn't the model just put more weight on them by themselves? Unless its because you want to make a nonlinear scaling of that variable. On a side note, isn't BMI a good example of poor feature design... 😀
I dont remember if i asked this already if I did sorry but it would be great if you could do a tutorial about mxnet/gluon. It is a advanced library that is good for advanced things.
Kind of the limit of the Bitter Lesson, as time approaches infinity is that any program can be written by a random number generator, if we have enough compute time, and a way to verify correctness. I think the cleaver algorithms are always filling in the gap before massive compute is able to perform this operation on its own. However, I still see Kaggles won on feature engineering, so I tend to assume that it is still a needed skill. At least for now.
Hi Jeff. I've recently subscribed and I honestly have to say you have the most comprehensive and easy to understand guides out there. Not to mention the fact that whenever there is an update to something, you make a new video explaining how to work with it. I tried getting in to machine learning just over a year ago and nobody at the time was able to actually explain anything apart from "download this, download that, if it doesn't work oh well" and would just go through the official tutorials without actually explaining how to do anything on your own. Your channel alone has given me the motivation to get started again and thank you so much for doing what you're doing!
Hello Leonard, thank you for the kind words. Glad the content is helpful, and yes, it is a lot of work keeping everything up to date.
Just wanted to leave a thank you Mr Heaton. I'm currently working on my bachelor thesis and your videos are a great help. Much appreciation.
Happy to help! Thank you for the note.
This is incredibly intuitive! Thanks
I've been following you for months, thank you for the free, well explained content!
Thanks!!
Amazing video Jeff ! The only thing you didn't tell us is if you then drop the source features to avoid collinearity or you just leave them along with the new features you created .... Or you perform PCA, VIF or Lasso after it to chose what to do?.... I loved the video concise and super useful!
Such a practical and helpful video, many thanks professor.
I read over your thesis comparing types of feature engineering vs machine learning models. Great stuff! Thx.
Thanks!
@@HeatonResearch Would standardizing or normalizing the input features give you better results? That one ratio had such a wide range.
@@jameswilliamson1726 I will often standardize/norm after applying these techniques. The techniques I use here are really to capture the interaction between underlying features. Then standardization/normlization on top solves range concerns.
Feature Engineering Explained! 😍
This is likely the best explanation on YT. Thx 🙏
I like Jeff's approach of giving us the big picture of he is talking about!
Thanks!
Fabulous explanation. In the early stages of my course ( MSc AI & Data Science ) and I find your channel very helpful. Thank you.
Thanks for this great information
This video and presentation is amazing. Thank you SO MUCH!! All the best!
Great, thank you.
You are welcome!
Super helpful! much appreciated!
Glad it helped!
excellent video of real practical use!
Love the energy!!!
Thanks! I also went a little crazy on video editing too. lol
this is amazing!
Very insightful. Thank you.
very informative
Really valuable content that is clearly explained! keep up the great work sir!
Awesome. Great explanation. Thank you 🙏
Very useful
At last, not another Data Science hijacker trying to prove themself on YT... Thank you.
This looks really fun to do!
great job!
Thanks!
Awesome video, thank you!
Very thanks for sharing
My pleasure
I found this video useful. Thanks!
Thank you :)
thank you! :)
When we do feature engineering, are we expecting that the new feature has a high correlation with the predicted values?
Yes for sure, so you must keep that in mind when evaluating feature importance. Generally, I leave the existing features in and let the model account for that (though some model types perform better with correlating fields removed).
Thia is really great and something out of box.
Can you please provide similiar techniques for NLP as well
Informative video as always. +1 like for my professor 👏
Thanks Jifan!
💛✌️ Thanks
You're welcome 😊
Awesome explanation! Thank you very much! Best regards from Ukraine!:)
thank you so much!!
Hi Jeff, what concepts should I look into to understand "Weighting" better? For instance at 9:41, you mention that if one values food more they might square it. Someone might cube it, someone might multiply it or add a coefficient of 2 or 5. These are all subjective.
For weighting when it comes to features in the stock market or econometrics (my specific application), one might have a feature that is GDP or inflation. I know for a fact that change in GDP (slope) and change in the change in GDP (slope of slope i.e., acceleration) are pretty important. My first problem, is that I found these two (change in GDP and GDP acceleration) simply through guess and check, and research papers. Is there a better method to this? Or should I focus on automating 'guess and check'? Secondly, sometimes the GDP features or inflation related features vary in importance to participants in the stock market. Perhaps right now (as of Oct 2022) investors might place more emphasis on inflation related features and so I might multiply inflation features by coefficient of 2 or square it. How would one deal with dynamic weighting? Or a simpler problem might be, how do you objectively select for weighting?
EDIT: I have come up with an idea, to add a coefficient to GDP or inflation based on social media mentions (sentiment), for instance. Thoughts on this and weighting in general?
Thanks so much! Love the video by the way!
Hi Jeff, would you have a link to your paper and the kaggle notebook that you showed?
Oh yeah, I should have linked that. I added it to the description, here it is too: arxiv.org/pdf/1701.07852.pdf
@@HeatonResearch Thanks a lot!
Thanks, great video! Any examples on using the shap package to additively decompose regression r^2 using shapley values?
I am novice. The model would figure out that relationship, then creating a new feature by dividing, multuplying something is worthy to do??
How should I perform Feature Engineering on anonymous variables? I cant put my domain knowledge on them
Thanks! Why would you e.g. square variables to make them more dominant in the model? Wouldn't the model just put more weight on them by themselves? Unless its because you want to make a nonlinear scaling of that variable.
On a side note, isn't BMI a good example of poor feature design... 😀
Can you try all different possible method to do this.
I dont remember if i asked this already if I did sorry but it would be great if you could do a tutorial about mxnet/gluon. It is a advanced library that is good for advanced things.
Currently researching Gluon for such a video.
@@HeatonResearch Nice.
@@HeatonResearch I always have a hard time getting it installed. You install guides are the best!!!!
Please show us how to customize StyleGan2 to for example generate a babyface or change the gender of someone in the image
Yes thinking about how to do something with that.
Would love to see a link to your paper?
Sure! Should have linked in the description. arxiv.org/abs/1701.07852
Thank you!
Can you address Sutton's Bitter Lesson as it applies here?
Kind of the limit of the Bitter Lesson, as time approaches infinity is that any program can be written by a random number generator, if we have enough compute time, and a way to verify correctness. I think the cleaver algorithms are always filling in the gap before massive compute is able to perform this operation on its own. However, I still see Kaggles won on feature engineering, so I tend to assume that it is still a needed skill. At least for now.
cunning man, does not fully say what really works and what I use by professionals