Ridge Regression From Scratch In Python [Machine Learning Tutorial]

We'll fully implement the ridge regression algorithm from scratch in python. Ridge regression is one of the most popular machine learning algorithms, and learning how it works can help you use it more effectively.
We'll cover what ridge regression is, including the theory and equation. Then we'll implement it in python and compare it to the reference implementation from scikit-learn. We'll end by learning how to calculate the optimal penalty lambda.
It's recommended to watch the video on linear regression before this one ( • Linear Regression Algo... ).
The full code and dataset are available here - github.com/dataquestio/projec... .
Chapters
00:00 - Introduction
00:16 - Ridge regression overview
08:40 - Ridge regression equation
16:53 - Python implementation of the ridge algorithm
26:11 - Finding the optimal penalty
-----------------------------
Join 1M+ Dataquest learners today!
Master data skills and change your life.
Sign up for free: bit.ly/3O8MDef

[@shank1er]

This is amazing. Better explanation than my professor.

[@grantkim9124]

How does this video not have a million views?

[@yedmitry]

Thanks for the video!

[@imildositoe1549]

I love the way the best alpha value is extracted

[@ling6701]

Awesome. Subscribed. Could you do a video on LASSO, and another on LogisticRegression?

[@ThinkRight2024]

Great explanation of ridge regression. Thank you. Just wondering what if a bunch of x variables has binary values (0,1), would I still have to standardize the data?
And also - could you also talk about applying similarity matrix in ridge regression… thank you.

[@colinwong3982]

Hi. Thanks for this amazing video. Is there any video about Lasso Regression?

[@Dataquestio]

Hi Colin - I don't have anything for lasso yet, but might work on it in the future. -Vik

[@kylmaz5782]

Hello. I want to implement the ridge regression method on a small dataset. but I want to get it by solving the model manually (by hand). How can I do it? I will be glad if you can help.

[@mikegher879]

Hey ! can i habe the codes for def ridge_fit() and def ridge_predict() ?
Thanks a lot!

[@SuperSumittanwar]

Hi vick!
Isnt it true that if explainabilty of target variable interms of equation is the goal then multicollinearity is an issue. However, if prediction is the goal then multicollinearity is not an issue.

[@Dataquestio]

Hi Sumit - in cases of multicollinearity, ridge regression will give you better predictions on out-of-sample data (test data) than linear regression.

[@rajeshmanjrekar3614]

great video, thank you for a great explanation of the ridge regression, however at the step of test_X @ B i am getting at error : "matrices are not aligned", i would be extremely greatful if you could send me a solution to this problem , thanks

[@Dataquestio]

It looks like your matrices don't have the right shapes to be multiplied. The full code is here, so you can check where the issue is in your code - github.com/dataquestio/project-walkthroughs/tree/master/ridge_regression .

[@skiprof]

@@Dataquestio This is the best instruction I have found on Ridge Regression, so thank you. However, I am having the same error (ValueError: matrices are not aligned). I looked over the code on github and have not been able to solve this problem. My dataframe has 9 variables (10 columns) x 80 rows. Any other suggestions to get past this error?

[@mquant001]

this youtube channel is pure BS

[@rajeshmanjrekar3614]

I am of the opinion that the sentence "B = np.linalg.inv(X.T@ X + penalty)@ X.T @ y" is creating this problem of matrix multiplication , kindly check....

[@learningwithfahad]

I had the same problem too being caused by NAs. Check your data for NAs.