What is great about this particular video is these concepts are explained well in many places like scattered dots , you connected the dots to paint the whole picture . an example for gradient descent included - very helpful .
Hi, Your video is helpful for beginners to understand the concept. One suggestion: In the very beginning of the video when you write the equation of your predicted line remember to mark it as y(cap) = mx(i) + c. It is not y(i) which is the actual data point.
I am a beginner and as a beginner I was struggling understand the gradient decent concept. I have seen many videos on gradient decent however all of them skipped explaining the derivative part however you explained it very well both (total and partial) with solution . Thanks!
Man, you've won my heart, you kept it so simple, best way of explaining Gradient Descent. Can you please help me in using learning rate in the equation and number of steps used in gradient descent with an example.
Hi @ranjiraj , at 21:37 u have given a wrong explanation in the partial derivative w.r.t C... d/dC (-C) will be -1 ,then why are you treating it as constant whereas in d/dc mx should be taken as constant.
As far as I know, gradient descent doesn't talk about solving 'm' and 'c' directly by putting them into two equations like the way you did it here..Because sometime the expression that we get after differentiation becomes so complex especially with logistic regression, random forest and other complex model in deep learning that solving 'm' and 'c' directly becomes extremely tricky and time complexity becomes very high...So Gradient descent talks about solving 'm' and 'c' by some trial and test method. Starting with some dummy value of 'm' and 'c' and putting those values in the equation of differentiation and check if the value of differentiation(say D0) for those 'm' and 'c' becomes 0 or close to 0. If not, then subtract that differentiation value(D0) from the old value of 'm' and 'c' and get a new 'm' and 'c' and again check if the value of differentiation (say D1) for those new 'm' and 'c' comes close to 0. Continue like this till you find that old 'm' - value of differentiation is very very close to 0. That 'm' becomes your actual 'm' and similarly the same thing to be done for 'c'.
This video was an intuitive way for understanding gradient descent for beginners. Anyways appreciate your time to quote about your understanding of GD.
@@RanjiRaj18 Good explanation. I think you have shown derivation for Ordinary Least Squares method. As far as Machine Learning is concerned it has to be slightly adapted.
He solved m and c in this case because it wasn't a very complex example. Only one independent variable. In a multiple linear regression it would have been much more complex.
will the sign ( direction ) for calculating m and b at last change, from addition to subtraction if we take "y-pred - y" instead of "y - y-pred" lke you have done in cost function ? i saw at few articles where this was mentioned but it was not clear.
Hi one question here: First derivative: xsquare =2x Second derivative = 2 ( replaced in same location) Third derivative = 0 While applying same in mean square error formula First derivative= I understood square to 2/n(•••) Second derivative: with respective to slope It should be 2/n I=1 to n (yi -xi -c) here I applied second derivative replacing. Since -mxi converts to -xi. But In ur explanation instead of replacing, you brought second derivative to starting as below: 2/n I= 1 to n -xi(yi-mxi-c) In the same way for intercept. One more At the end , what happened to 2/n? Please correct me if I am wrong.
2/n Σi=1 to n -xi(yi-mxi-c) this comes from chain rule watch this part carefully again in the video, this (y-mxi-c) is the result of (yi-mxi-c)^2 and now since we want to differentiate wrt to slope m again you take the derivative now you treat the y and c as constants and what's left is -mxi so you get -xi that's what you get and you multiply with this(y-mxi-c). 2/n is a constant say for any number, n=5; 2/5 =constant you eventually equate it to zero so it vanishes away. Hope now you understand!
That is a explanation. I have one question, where would the learning rate be actually used in computation. Like in your numerical example. We found the outputs and calculated corresponding m and c. How does the learning rate is catered. Secondly, when we multiply learning rate with derivative, what does it gives us ?
i have this video over and over again, it the most satisfying video i have seen, in as much as gradient decent is concer, but i have questions, 1 what happen to the 2 tha became the multiple of the function as chain rules implies, then what happen to the no in the cost function. i know its mean squared errored thing. in my small assumption either of the values cant be thrown away just like that mathematically. please help with explanation.
between the timestamp 21:39 to 21:45 you told that partial derivative of y(i)-mx-c with respect to c is 0 so only take minus sign which is wrong it will be -1 because of here c is not constant.
Hi Ranji Sir, i have a doubt. In 17:00 you have mentioned that we need to take derivative because we have two variables. And mentioned variables as x and c. But I think you were supposed to say m and c. later in 17:27 you mentioning about two parameters m and c. Please verify whether it correct or not. If i have pointed out wrong please apologize me.
The explanation of mathematical formula is absolutely fantastic. The explanation was about with single feature. But if we have multiple feature, what to be changed in the equation? Can you please let us know that. Thanks very much, and we will love to see this kind of videos shortly.
In case of multiple features or weights we have to conisder them individually by taking the partial derivtaive. This video is just a general idea of gradient descent. Hope it answers your question.
Sir my maths is quite weak i want to start my career in data science i know that i can make my math strong but how should i start to learn maths for data science
Hi Ranji sir, I wanted to ask that if there are if our line is of the form M1*(feature1) + M2*(feature2).... Mn*(feature n) + c, do we have to follow same steps and calculate dJ/dm for all M1, M2...Mn?
If I understood your question correctly then: When? learning rate is made use for convergence, it should not be neither too large nor too low just optimal, so that your traning process is complete. How? You can use learning rate schedule or can use optimizers like ADAM.
When there are relatively larger coefficients in your model, taking total derivate would be a diffuícult task and also to estimate the optimal parameter. Partial derivatives reduces the workload by keeping one parameter as constant and determine the other.
Thanks you so much but I have small clarification regarding differentiate. why we are differentiate with respect to m , c & why we should not differentiate with respect to x to find out the value of y..
J = 1/(2*m) * sum (h(x)-y)^2. being h(x) the hipotesis and y the accurate value... at 3:37 you got them mixed up right? damn man.. no wonder people get confused
For notes👉 github.com/ranjiGT/ML-latex-amendments
You just helped me understand hundreds of web pages that talked about topics with no order. Thank you
What is great about this particular video is these concepts are explained well in many places like scattered dots , you connected the dots to paint the whole picture . an example for gradient descent included - very helpful .
Thank You for your valuable feedback 😊
You are such a great teacher. Concepts are clearly explained beginning with the basics and slowly easing into the most advanced level. Thank you
You're very welcome!
Hi, Your video is helpful for beginners to understand the concept. One suggestion: In the very beginning of the video when you write the equation of your predicted line remember to mark it as y(cap) = mx(i) + c. It is not y(i) which is the actual data point.
The best and clear explanation I've ever listened about Gradient Descent. Keep up the good work!🙌
Awesome, thank you!
My god you are perfect I think your work should reach more audience your best and clear than the renowned ML yputubers. Applause Ranji
Your hard work made the concept very easy to grasp. Kudos.......
its a very good description, The way you teach is humble and appreciatable.
Thank you very much!
Everything is so easy on this channel, great work Man!
Glad you like them!
Hey thank you so much for this content since I started studying regression using your videos , I became huge fan of yours
Thank you for your comment. Glad you like it ;)
Sir thank you very much, this has been so helpful since my course will only get tougher from here onwards and u helped me understand the basics
I am a beginner and as a beginner I was struggling understand the gradient decent concept. I have seen many videos on gradient decent however all of them skipped explaining the derivative part however you explained it very well both (total and partial) with solution . Thanks!
Great to hear!
Man, you've won my heart, you kept it so simple, best way of explaining Gradient Descent. Can you please help me in using learning rate in the equation and number of steps used in gradient descent with an example.
Hi @ranjiraj , at 21:37 u have given a wrong explanation in the partial derivative w.r.t C...
d/dC (-C) will be -1 ,then why are you treating it as constant whereas in d/dc mx should be taken as constant.
this might be the most underrated explanation on youtube
This is what I pay my internet bill for! Thanks a lot!
When my ML teacher teaching this , I felt I am learning some rocket science ,but you are teaching it felt very easy , thank you Sir😊
Glad to hear that
I was unable to understand this topic tried many videos but this was the most useful video thankss
As far as I know, gradient descent doesn't talk about solving 'm' and 'c' directly by putting them into two equations like the way you did it here..Because sometime the expression that we get after differentiation becomes so complex especially with logistic regression, random forest and other complex model in deep learning that solving 'm' and 'c' directly becomes extremely tricky and time complexity becomes very high...So Gradient descent talks about solving 'm' and 'c' by some trial and test method. Starting with some dummy value of 'm' and 'c' and putting those values in the equation of differentiation and check if the value of differentiation(say D0) for those 'm' and 'c' becomes 0 or close to 0. If not, then subtract that differentiation value(D0) from the old value of 'm' and 'c' and get a new 'm' and 'c' and again check if the value of differentiation (say D1) for those new 'm' and 'c' comes close to 0. Continue like this till you find that old 'm' - value of differentiation is very very close to 0. That 'm' becomes your actual 'm' and similarly the same thing to be done for 'c'.
This video was an intuitive way for understanding gradient descent for beginners. Anyways appreciate your time to quote about your understanding of GD.
@@RanjiRaj18 Good explanation. I think you have shown derivation for Ordinary Least Squares method. As far as Machine Learning is concerned it has to be slightly adapted.
He solved m and c in this case because it wasn't a very complex example. Only one independent variable. In a multiple linear regression it would have been much more complex.
Truest your the best. You solve my long time machine learning challenge.
Very good explanation. It would've been good if you could've explained the usage of learning rate usage to find a minimum point.
perfect video for core concept understanding , amazing.. I love the explanation.. thankyou so much
Great explanation with an example.
This is the way to explain such concepts.
Thank you for the comment. Happy Learning!
will the sign ( direction ) for calculating m and b at last change, from addition to subtraction if we take "y-pred - y" instead of "y - y-pred" lke you have done in cost function ? i saw at few articles where this was mentioned but it was not clear.
Really amazing, thank you so much sir, keep rocking
Always welcome
Thanks for the clear explanation sir
Well explained bro ❤ just bring another video for statistics and linear algebra 🎉
simple and useful lecture.. thanks
You are welcome
Very good explanation of gradient descent
killer explanation !! amazingly amazing !! thank you bro
explanation is simply awesome.....
Great video man. Loved it.
At 20.37, you discarded n/2. I would like to know why?
Thank you so much, I got a clear picture of the topic now.
You are welcome!
Absolutely the best explanation
If I got it your last example is analytical solution, but it couldn't been done everytime, then we use iterative solution with alpha learning rate?
Thanks a lot sir! It was really helpful. Excellent explaination.
Thank you! Wish me luck on exam about it!
Good Luck 👍
very nicely explained Bro
Glad you liked it
you earned a subscriber my man
Hi one question here:
First derivative: xsquare =2x
Second derivative = 2 ( replaced in same location)
Third derivative = 0
While applying same in mean square error formula
First derivative= I understood square to 2/n(•••)
Second derivative: with respective to slope
It should be
2/n I=1 to n (yi -xi -c) here I applied second derivative replacing.
Since -mxi converts to -xi.
But In ur explanation instead of replacing, you brought second derivative to starting as below:
2/n I= 1 to n -xi(yi-mxi-c)
In the same way for intercept.
One more
At the end , what happened to 2/n?
Please correct me if I am wrong.
2/n Σi=1 to n -xi(yi-mxi-c) this comes from chain rule watch this part carefully again in the video, this (y-mxi-c) is the result of (yi-mxi-c)^2 and now since we want to differentiate wrt to slope m again you take the derivative now you treat the y and c as constants and what's left is -mxi so you get -xi that's what you get and you multiply with this(y-mxi-c). 2/n is a constant say for any number, n=5; 2/5 =constant you eventually equate it to zero so it vanishes away. Hope now you understand!
Thanks for this tutorial sir..made it very easy and simple.
your videos are really nice, good content and presentation...keep it up sir.
That is a explanation. I have one question, where would the learning rate be actually used in computation. Like in your numerical example. We found the outputs and calculated corresponding m and c. How does the learning rate is catered. Secondly, when we multiply learning rate with derivative, what does it gives us ?
Sir linear regression is not used for classification as you said in satrting of video while explaining
Thanks so much for the video. It helped me a lot.
can you please tell me why the curve is much sharp when you draw the graph with respect to j and c ? please tell
i have this video over and over again, it the most satisfying video i have seen, in as much as gradient decent is concer, but i have questions, 1 what happen to the 2 tha became the multiple of the function as chain rules implies, then what happen to the no in the cost function. i know its mean squared errored thing. in my small assumption either of the values cant be thrown away just like that mathematically. please help with explanation.
between the timestamp 21:39 to 21:45 you told that partial derivative of y(i)-mx-c with respect to c is 0 so only take minus sign which is wrong it will be -1 because of here c is not constant.
sir u explained so well
great explainnation
Thank you so much sir for such a perfect explanation....🙏🙏👏👏👏
Thankyou sir ,
Make more videos on machine learning concepts .
Hi Ranji Sir, i have a doubt. In 17:00 you have mentioned that we need to take derivative because we have two variables. And mentioned variables as x and c. But I think you were supposed to say m and c. later in 17:27 you mentioning about two parameters m and c. Please verify whether it correct or not. If i have pointed out wrong please apologize me.
Yes you are correct we have to take derivative wrt m and c
really nice approch to teach
thank you sirji
one of the best
very nice explanation
excellent video
thank you so much sir
Thankyou sir 😊
Thanks bro.
Thanks ...very well explained
Best explanation sir
why do we set the derivative equal to 0? i mean the gradient at the minima might not be equal to zero for all curves
Nicely explained.
keep up the good work !
what about when someone is working with the learning rate?
The explanation of mathematical formula is absolutely fantastic. The explanation was about with single feature. But if we have multiple feature, what to be changed in the equation? Can you please let us know that. Thanks very much, and we will love to see this kind of videos shortly.
In case of multiple features or weights we have to conisder them individually by taking the partial derivtaive. This video is just a general idea of gradient descent. Hope it answers your question.
Lovely explanation
Thanks Buddy :)
thank you!!!
It was very helpful, in writing a thesis. Could you also indicate some bibliography for citations
What should i do if i am to apply learning rate of some value?
Sir my maths is quite weak i want to start my career in data science i know that i can make my math strong but how should i start to learn maths for data science
Hello Maria, you can refer to websites like www.mathsisfun.com/ to learn the basics. Hope it helps!
@@RanjiRaj18 thanks sir
@@RanjiRaj18 Sir its very low level mathematics I am in sybsc it
hi, sir do you have the python code using tensor flow or do you have any recordings of ML using TF
great explaination
Hi Ranji sir, I wanted to ask that if there are if our line is of the form M1*(feature1) + M2*(feature2).... Mn*(feature n) + c, do we have to follow same steps and calculate dJ/dm for all M1, M2...Mn?
Yes, one by one
What type is it batch gradient?
can you please explain me why we are squaring the step at 4.36 . everything is clear to me except this one squaring step I cant able to understand..
Amazing lecture. x^n, will not have a 3rd order derivate to be 0, it will be n+1 order derivate.
Well explained
dear sir,
I still cannot connected what is the purpose we do m = m - lambda * dJ/dm and c = c - lambda * dJ/dC
Yess .. exactly....plz explain the proof of these 2 equations
Thank you so much..quick question, when/how do we use the learning rate in this regard?
If I understood your question correctly then:
When? learning rate is made use for convergence, it should not be neither too large nor too low just optimal, so that your traning process is complete.
How? You can use learning rate schedule or can use optimizers like ADAM.
@@RanjiRaj18 Yes, Alright thank you so much. Your vid was really helpful.
is it a sweet, in the middle of a hyperplane?
can this method work for an equation with multiple slopes?
Why do we need partial derivative when we have the total derivative?
When there are relatively larger coefficients in your model, taking total derivate would be a diffuícult task and also to estimate the optimal parameter. Partial derivatives reduces the workload by keeping
one parameter as constant and determine the other.
Linear regression for classfication at 0:20???
Thanks you so much but I have small clarification regarding differentiate. why we are differentiate with respect to m , c & why we should not differentiate with respect to x to find out the value of y..
Because m, c are the weights that we want to determine which will give the best equation for curve fitting.
@@RanjiRaj18 thanks you so much for spending time to respond to my comment..
How did the 2/n gone from equation when dJ/dm and dj/dc was assigbed to 0
2/n is a constant say you take n =5 so it becomes 2/5 so it's derivative is zero
@@RanjiRaj18 Thank you Ranji
Excellent...
So gradient descent is our cost function to calculate the error
Thanks Bro :)
bro please ans this question why we are taking summation of c in one equation and not in other ---> one is 5C why
Plase ans
thanks a lot..... :)
🙂👍 nice like that
Which level of maths is required 11th and 12th or degree level mathematics ?
Personally both Derivatives, differential equations, Matrices and vector concepts
@@RanjiRaj18 thanks
why at the end multiply the 2 nd equation with 5
To make equation balance on both sides for cancellation. Those are basic algebraic rules.
Sir can you recommend a book for machine learning with mathematical background please
You can refer the book by `Tom Mitchell`
@@RanjiRaj18 sir please what is book name and if you share pdf link it will be better
@@shafiqahmad9057 You can check on google it is open source
@@RanjiRaj18 thank you for very fast respomse
sir explane krne ke baad thoda side ho jaya kijiye screenshot lena rehta hai
J = 1/(2*m) * sum (h(x)-y)^2. being h(x) the hipotesis and y the accurate value... at 3:37 you got them mixed up right? damn man.. no wonder people get confused
Sorry bro, Not clear what you saying..