Mini Batch Gradient Descent | Deep Learning | with Stochastic Gradient Descent
Vložit
- čas přidán 27. 07. 2024
- Mini Batch Gradient Descent is an algorithm that helps to speed up learning while dealing with a large dataset.
Instead of updating the weight parameters after assessing the entire dataset, Mini Batch Gradient Descent updates weight parameters after assessing the small batch of the dataset. Thus we can make much progress before our model sees the entire dataset. Thus the learning can be very fast.
In the video, we also saw Stochastic Gradient Descent, which updates the weight parameter after evaluating every data point or data record. Stochastic Gradient Descent has its own disadvantages as well, which is overcome by the Mini Batch Gradient Descent.
So, in practice, we don't use Stochastic Gradient Descent but we use Mini Batch Gradient descent while dealing with large datasets. And we use simple, Batch Gradient Descent while dealing with small datasets.
➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖
✔ Improving Neural Network Playlist: • Overfitting and Underf...
➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖
✔ Complete Neural Network Playlist: • Neural Network python ...
✔ Complete Logistic Regression Playlist: • Logistic Regression Ma...
✔ Complete Linear Regression Playlist: • How Neural Networks wo...
➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖
Timestamp:
0:00 Agenda
0:38 When Gradient Descent will Fail
2:24 Mini Batch Gradient Descent Deep Learning
3:31 Drawback of Mini Batch Gradient Descent
5:47 Stochastic Gradient Descent
6:30 When to use Mini Batch Gradient Descent
➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖
This is Your Lane to Machine Learning ⭐
➖➖➖➖➖➖➖➖➖➖➖➖➖➖➖
Subscribe to my channel, because I upload a new Machine Learning video every week: / @codinglane
If you found this video helpful, then hit the *_like_* button👍, and don't forget to *_subscribe_* ▶ to my channel as I upload a new Machine Learning Tutorial every week.
Amazing helpful video!
amzing, deeply explained. thanks
This is by far the best video on introduction to optimizers.
Very precise, articulated and clears all the doubts.
Thanks a lot brother !
Glad it helped you 😇
Great job!
very good explanation. u need more views
Very nice Explanation. Super
Oh wow. When I was start learning ML, last year, your linear regression was one of the vids I first watched. Now, I'm a data scientist, you're still uploading high quality vids. Thank you! Hopefully we could get to see LSTMs and Transformers in the future. :P good day.
Wow... Really good to know this. Thank you for sharing your story! And Yes I will be uploading videos on LSTM.
@@CodingLane Wow nice! Can I make a suggestion? Maybe in the future you can include weight initialization like Xavier and He Norm. Those topics tend to be ignored because the computer is basically covering those, (I'm guilty of that :P) without knowing the reason behind it, e.g the disadvantages of weight initialization with 0 value.
@@arvinflores5316 Thank you for giving this suggestion. I will definitely consider making videos on these topics.
very good explanation! well done!
Thank you!
very informative and precise.
Thank you!
Brilliant explanation... Keep it up!
Thank you!
you got a new sub bro, good video
Thank you so much for your videos.
You're Welcome. Glad you like them! 😇
thx
Amazing dude. Keep it up.
Thank You So Much !!
👌
Great job
Thank you!
ty
Welcome!
good video! but I have only one question: where does the noise come from, that you mentioned at 5:11?
Thats how the loss changes when we have more number of features
good boy
So from what I understand,
in Mini Batch Gradient Descent, model will train 1st mini batch, update the weights and then those updated weights will be used to train the 2nd mini batch, update and then the 3rd mini batch,..., till the last mini batch (1 epoch), then the last mini batch updated weight will be again used on 1st mini batch during 2nd epoch and so on?
Do correct if wrong.
You are correct on your understanding Abhinav. I would just like to correct the words. You can say that the updated weights after training on any mini-batch is used to propoagate forward, and then they are updated again in backward propagation. Eg, randomly initialise weights at the beginning. Propagate forward (perform forward propagation) using 1st mini batch, then perform backward propagation, then update weights. Use those updated weights to perform forward propagation using 2nd minibatch, then backward propagation, update weights again and so on.
@@CodingLane Thanks :)
Your videos are helpful, Can you suggest a good book on same...
Hi... I don’t refer any book so can’t suggest you any. Although you can search for good books on ML online. I once found an article which showed top 10 books for learning ML.
@@CodingLane Ok ! Thanks!...But have topics like batch normalisation and standard network like LeNet, AlexNet, AGG, GoogleNet in detail
@@EEDNAGELIVINAYAKSHRINIWAS Have you read the original research papers for these? I think you can learn about these in their research papers only.
@@CodingLane Ok..can you mail me on my id lets discuss separately on some projects
Hi Vinayak, you can mail me on codeboosterjp@gmail.com with your query. I will see what i can do to help.