Regularization Part 2: Lasso (L1) Regression
Vložit
- čas přidán 5. 06. 2024
- Lasso Regression is super similar to Ridge Regression, but there is one big, huge difference between the two. In this video, I start by talking about all of the similarities, and then show you the cool thing that Lasso Regression can do that Ridge Regression can't.
NOTE: This StatQuest follows up on the the StatQuest on Ridge Regression:
• Regularization Part 1:...
For a complete index of all the StatQuest videos, check out:
statquest.org/video-index/
If you'd like to support StatQuest, please consider...
Buying The StatQuest Illustrated Guide to Machine Learning!!!
PDF - statquest.gumroad.com/l/wvtmc
Paperback - www.amazon.com/dp/B09ZCKR4H6
Kindle eBook - www.amazon.com/dp/B09ZG79HXC
Patreon: / statquest
...or...
CZcams Membership: / @statquest
...a cool StatQuest t-shirt or sweatshirt:
shop.spreadshirt.com/statques...
...buying one or two of my songs (or go large and get a whole album!)
joshuastarmer.bandcamp.com/
...or just donating to StatQuest!
www.paypal.me/statquest
Lastly, if you want to keep up with me as I research and create new StatQuests, follow me on twitter:
/ joshuastarmer
#statquest #regularization
If you want to see why Lasso can set parameters to 0 and Ridge can not, check out: czcams.com/video/Xm2C_gTAl8c/video.html
Support StatQuest by buying my book The StatQuest Illustrated Guide to Machine Learning or a Study Guide or Merch!!! statquest.org/statquest-store/
One of the best explanation of Ridge and Lasso regression I have seen till date... Keep up the good work....Kudos !!!
Thanks! :)
I am eternally grateful to you and those videos!! Really saves me time in preparing for exams!!
Happy to help!
Love how you keep these videos introductory and don't go into the heavy math right away to confuse;
Love the series!
Thank you!
That "Bam???" cracks me up. Thanks for your work!
:)
Hi, I can't thank you enough for explaining the core concepts in such short amount of time. Your videos help a lot! My appreciations are beyond words.
Thank you!
The difference between BAM??? and BAM!!! is hilarious!!
:)
@@statquestCan you please explain how the irrelevant parameters "shrink"? How does Lasso go to zero when Ridge doesn't?
@@SaiSrikarDabbukottu I show how it all works in this video: czcams.com/video/Xm2C_gTAl8c/video.html
Every time I think your video subject is going to be daunting, I find you explanation dispel that thought pretty quickly. Nice job!
Hi man, really LOVE your videos. Right now I'm studying Data Science and Machine Learning and more often than not your videos are the light at the end of the tunnel, sot thanks!
My teacher is 75 years old, explained us Lasso during one hour , without explaining it. But this is a war I can win :), thanks to your efforts.
I love it!!! Glad my video is helpful! :) p.s. I got the joke too. Nice! ;)
Why is this scenario many times the reality? Also, I check StatQuest's vids very often to really understand the things. Thanks @StatQuest
Don't think your Monty Python reference went unnoticed
(Terrific and very helpful video, as always)
Thanks so much!!! :)
Oh it absolutely did. And it was much loved!
Some video ideas to better explain the following topics:
1. Monte Carlo experiments
2. Bootstrapping
3. Kernel functions in ML
4. Why ML is black box
OK. I'll add those to the to-do list. The more people that ask for them, the more I'll priority they will get.
@@statquest That is great! keep up the great work!
@@statquest yes we need it please do plsssssssssssssssssssssssssssssssss
plsssssssssssssssssssssssssssssssssssssssssssssss
Bootstrapping is explained well in Random Forest video.
Do it for us... thanks good stuff
I am so happy to easily understand these methods after only a few minutes (after spending so many hours studying without really understanding what it was about). Thank you so much, your videos are increadibly helpful! 💯☺
Great to hear!
Josh - as always your videos are brilliant in their simplicity! Please keep up your good work!
Thanks, will do!
Thank you, Josh, for this exciting and educational video! It was really insightful to learn both the superficial difference (i.e. how the coefficients of the predictors are penalized) and the significant difference in terms of application (i.e. some useless predictors may be excluded through Lasso regression)!
Double BAM! :)
This channel is pure gold. This would have saved me hours of internet search... Keep up the good work!
Thank you! :)
Good video, but didn't really explain how LASSO gets to make a variable zero. What's the difference between squaring a term and using the absolute value for that?
Intuitively, the closer slope gets to zero, the square of that number becomes insignificant compared to the increase in the sum of the squared error. In other words, the smaller you slope, the square gets asymptotically close to 0 because it can't outweigh the increase in the sum of squared error. In contrast, the absolute value adds a fixed amount to the regularization penalty and can overcome the increase in the sum of squared error.
@@theethatanuraksoontorn2517 Maybe this discussion on stack-exchange will clear things up for you: stats.stackexchange.com/questions/151954/sparsity-in-lasso-and-advantage-over-ridge-statistical-learning
@@statquest Thanks for reading the comments and responding!
@@programminginterviewprep1808 I'm glad to help. :)
@@statquest I didn't reply before, but the answer really helped me a lot, with basic machine learning and now artificial neural networks, thank you very much for the videos and the replies :D
Bam! I appreciate the pace of the videos. Thanks for doing this.
Thanks! :)
Explained in a very simple yet very effective way! Thank you for your contribution Sir
Hooray! I'm glad you like my video. :)
Your intro songs reminds me of Pheobe from the TV show "Friends", and the songs are amazing for starting the videos on a good note, cheers!
You should really check out the intro song for this StatQuest: czcams.com/video/D0efHEJsfHo/video.html
This is brilliant. Thanks for making it publicly available
You're welcome! :)
Thx very much. Clear explanation for these similar models. Great video I will conserve forever
I am eternally grateful to you. You've helped immensely with my last assessment in uni to finish my bachelors
Congratulations!!! I'm glad my videos were helpful! BAM! :)
Yeahhhh!!! I was the first to express Gratitude to Josh for this awesome video!! Thanks Josh for posting this and man! your channel is growing.. last time, 4 months ago it was 12k. You have the better stats ;)
Hooray! Yes, the channel is growing and that is very exciting. It makes me want to work harder to make more videos as quickly as I can. :)
@@statquest please keep on going... You are our saviour
Came here because I didn't understand it at all when my professor lectured about LASSO in my university course... I have a much better understanding now thank you so much!
Awesome!! I'm glad the video was helpful. :)
So easy to understand. And I like the double BAM!!!
Thanks!
Your videos make it so easy to understand. Thank you!
Thank you! :)
Thanks for posting, my new favourite youtube channel absolutely !!!!
Wow, thanks!
Incredible great explanations of regularization methods, thanks a lot.
Thanks! :)
The other day, I had homework to write about Lasso and I struggled.. wish I had seen this video a few days earlier.. Thank you as always!
The beginning songs are always amazing hahaha!!
Awesome! :)
I came for the quality content, fell in love with the songs and bam.
BAM! :)
I really appreciated the inclusion of swallow airspeed as a variable above and beyond the clear-cut explanation. Thanks Josh. ;-)
:)
Me too!
Hooray!!!! excellent video as always
Thank you!
Hooray, indeed!!!! Glad you like this one! :)
You have a gift for teaching! Excellent videos!
Thanks! :)
this is awesome thank you so much for this u explained it so well . I will recommend this video to every one I know who is interested . I also watched your lasso video and it was just as good thank you
Thank you very much! :)
Dude you are an absolute lifesaver! keep it up!!!
Hooray! I'm glad I could help. :)
Thank you so much for the video !
I have watched several your videos and I prefer to watch your video first then see the real math formula. When I did that, the formula became so easier and understandable!
For instance, I don't even know what does 'norm' is, but after watching your video then it would be very easy to understand!
Awesome! I'm glad the videos are helpful. :)
Thank you so much for these videos you are a literal godsend. You should do a video on weighted least squares!!
Great video, clear explanation, loved the Swallows reference! Keep it up! :)
Awesome, thank you!
Thanks a lot! Amazing explanation! Please, continue the great work and add more on statistics, probability in general and machine learning in particular. Sinse Data Science suppose to have a great future, I am certain that your channel also will prosper a great deal!
Thank you! :)
Airspeed of swallow lol. These videos are really helping me a ton, very simply explained and entertaining as well!
Glad you like them!
Thanks! I finally understand how they shrink parameters!
a man of his word...very clearly explained!
Thank you! :)
Excelent video Josh! Amazing way to explain Statistics Thank you so much! Regards from Querétaro, México
Muchas gracias! :)
Amazing video, explanation is fantastic. I like the song along with the concept :)
Bam! :)
Thank you once again Josh!
bam!
Fantastic videos - very well explained!
Thank you! :)
Ah! A triple THANKSSSS!!!!. I finally got what they are really doing.
Hooray! :)
Hi Josh, Thanks for clear explanation on regularization techniques. very exciting. God bless for efforts.
Glad you enjoyed it!
Seriously the best videos ever!!
Thanks!
The best explanation ever.
Thankyou Sir ! Great Help.
I love your style of explaining! You leave enough time for anyone to take in all information while talking. Sometimes it feels like you are trying to teach little kids, but it actually just works. I often watch other teaching videos and can't remember most of it afterwards, but I can remember almost everything that you are saying after the first time watching. Amazing job!
I have one question though. You were saying that the regression model in the beginning had low bias and high variance. Does it not have high bias? As far as I know bias represents the expected generalization (or test) error, if we were to fit a very large training set. If we fit that simple model to a lot of data, the generalization error would be rather high, because it could not capture the true patterns in the data.
I'm glad you like the videos! In ML, there are specific meanings for bias and variance that are a little bit different from what you are using and I explain in this StatQuest: czcams.com/video/EuBBz3bI-aA/video.html
Just love the way you say 'BAM?'.....a feeling of hope mixed with optimism, anxiety and doubt 😅
:)
Me and my friend are studying. When the first BAM came, we fell for laught for about 5min. Then the DOUBLE BAM would cause a catrastofic laughter if we didn't stop it . I want you to be my professor please!
BAM! :)
Brilliant explanation
didnt need to check out any other video
Thank you!
Great video! I finally understand!
Thank you so much for making these videos! Had to hold a presentation about LASSO in university.
I hope the presentation went well! :)
@@statquest Thx. It did :)
Always amazing videos.
Thank you!
Hi Josh! I am a big fan of your videos and it is clearly the best way to learn machine learning. I would like to ask you if you will be uploading videos relating to deep learning and NLP as well. If so, that will be awesome. BAM!!!
Right now I'm finishing up Support Vector Machines (one more video), then I'll do a series of videos on XGBoost and after that I'll do neural networks and deep learning.
StatQuest with Josh Starmer Thanks Josh for the updates. I’ll send you request at Linkedin.
Keep it up man. Awesome content.
Thanks! :)
Hi thanks I think the videos are great and also I like your songs. You are very talented.
In looking at this video I was thinking that Lasso regression can be used as a form of variable selection. Is this a good idea? So basically at first you include all the predictors then the lasso will tell which variable you need to get rid off. Does this make sense?
Statquest is like Marshall Eriksen from HIMYM teaching us stats. BAM? Awesome work Josh.
Thanks!
Great video! The topic is really well explained
Thank you!
so incredible, so well explained
Thanks!
Wow! so easy to understand this! Thanks very much!
Thanks!
Thanks for the Video. They make difficult concepts seem really easy..
Thank you! :)
@@statquest Can u make a similar video for LSTM?
Great people know subtle differences which is not visible to common eyes
love you sir
Thanks!
Hi. Your videos are so helpful. I really appreciate you spend time doing them.
I have one question related to this video: Is the result of Lasso Regression sensitive to the unit of variables?
For example in the model: size of mice = B0 + B1*weight + B2*High Fat Diet + B3*Sign + B4*AirSpeed + epsilon
Suppose the original unit of weight in the data is gram. If we divide the weight by 1,000 to get unit in kilogram, is the Lasso Regression different?
As I understand, the least square estimated B1-kilogram should be 1,000 times higher than the B1-gram. Therefore, B1-kilogram is more likely to be vanished in Lasso, isn't?
Amazing! Thank you so much for this!
Thanks!
Thank you for uploading this video. Can you upload a video explaining the difference between Lasso and Group Lasso? Thanks again.
I enjoy the content and your jam so much! '~Stat Quest~~'
Thanks!
NOBODY IS GOING TO TALK ABOUT THE EUROPEAN / AFRICAN SWALLOW REFERENCE ????are you all dummies or something ? It made my day. Plus, video on top, congratulation. BAMM !
bam!
A StatQuest a day, keeps Stat fear away!
I love it! :)
wonderfully explained
Thank you! :)
Magnificent video
Thanks!
Thanks a lot for the explanation !!!
You are welcome!
awesome your explanation just simplifies everything
request to make videos on rest of the algorithms as well
thank you
I'm working on them :)
This was gold!
Thank you!
Love this song bro!
Thanks!
You are the best! I understand it now!
Thanks!
Hahaha.. That moment you said BAM??? I laughed out loud 🤣🤣🤣
:)
Thanks Josh!
You're welcome! :)
BAM!! clearly explained!!
Hooray! I'm glad you like the video. :)
love the work, i remember reading books about linear regresion, when they spent like 5 pages for these 2 topics but i still have no clue what they really do =))
Glad it was helpful!
Love the fact that you reply to every single comment here in YT haha
Very good video. You help me alot !!!!
Thanks! :)
Million BAM for this channel 🎉🎉🎉
Thank you!
One more use case of Ridge/Lasso regression is 1) When data points are less 2) High Multicollinearity between variables
Great....please continue to learn other models...thank you so much.
Thanks!
You're great!! Thank You
You're welcome!!! :)
me: wathcing these videos in full panic
video: plays calming music
me: :)
bam! Good luck! :)
This is dope fam!
:)
Nice explanation!
Thanks!
keep up ur work bro .
Thanks, will do!
Great explanation
Thank you! :)
I prefer the intro where is firmly claimed that StatQuest is bad to the bone. And yes I think this is fundamental.
That’s one of my favorite intros too! :)
But I think my all time favorite is the one for LDA.
Yes I agree! Together these two could be the StatQuest manifesto summarising what people think about stats!
So true!
Great work, Thank you Josh,
I'm trying to connect ideas from different perspectives/angles, Does the lambda here somehow related to Lagrange multiplier ?
I'm not sure.
awesome video
Thanks!
Best youtube channel
Thank you! :)