Tutorial 9- Drop Out Layers in Multi Neural Network
Vložit
- čas přidán 23. 07. 2019
- After going through this video, you will know:
Large weights in a neural network are a sign of a more complex network that has overfit the training data.
Probabilistically dropping out nodes in the network is a simple and effective regularization method.
A large network with more training and the use of a weight constraint are suggested when using dropout.
Below are the various playlist created on ML,Data Science and Deep Learning. Please subscribe and support the channel. Happy Learning!
Deep Learning Playlist: • Tutorial 1- Introducti...
Data Science Projects playlist: • Generative Adversarial...
NLP playlist: • Natural Language Proce...
Statistics Playlist: • Population vs Sample i...
Feature Engineering playlist: • Feature Engineering in...
Computer Vision playlist: • OpenCV Installation | ...
Data Science Interview Question playlist: • Complete Life Cycle of...
You can buy my book on Finance with Machine Learning and Deep Learning from the below url
amazon url: www.amazon.in/Hands-Python-Fi...
🙏🙏🙏🙏🙏🙏🙏🙏
YOU JUST NEED TO DO
3 THINGS to support my channel
LIKE
SHARE
&
SUBSCRIBE
TO MY CZcams CHANNEL
krish sir just one thing to say...i too teach myself sometimes to school children,the thing is the effort you are putting in making these videos at free of charge is commendable...May god bless you sir..I am gaining confidence too after seeing ur videos and and thus becoming a data scientist
You are the mentor every aspiring data scientist needs, Thanks!!
Love the Deep Learning Series. Great Learning !!
This deeplearning series is extremely good.
Thanks for putting your efforts in making these in-depth videos which clarifies concepts in detail. Your videos are helping students like me who are very new to the ML and AI field.
I am Msc. student from Ethiopia, Really to tell you the fact I have learnt a lot from your videos. May God bless your mind!!
Just I can see ur face is full of happiness when u explains a concept
I guess u r like 🙏🙏
You have a knack of making things short and simple and easy to grasp :)
I found it extremely useful, easier to understand than many known experts
Thanks for the sessions... These are precise and organized...
Really Like the way you explain! I have just completed Udemy Bootcamp and you are definitely reinforcing what I have learned. Keep up the good work!
Great service. Amazing Explanation!!
Krish Sir you are my favorite Teacher...your lessons and explanation's are simple and easy to understand , me like B grade student also can understand the concepts. Thank you Sir.
That's the good video Krishna, I never thought about the random forest playing a similar mechanism when the first time I was studying dropout. good, you've cleared my concept with this video. Thanks!
Great stuff. But I have to listen several times to understand given our different dialects. Much appreciation for your work and explanations!! Excellent!
The effort in these Videos !!!
Thanks Krish !!!
Great as always! Thank you :)
Such awesome content and explanations!!!
You explain very good! Thank you!
I watched 10 videos but yet i didn't code anything, but i am sure whenever I will code. I will be perform in more clearly.because these are videos are focusing on more basics and defining the more depth of ANN. Thank you so much sir. 🥰🥰😘🇮🇳🇮🇳
Great explanations, thank you very much sir
Thank you. Much easier to understand than the one by Andrew Ng.
But can't ignore the fact , that he is God in AI
Do u take and finish Andrew Ng course?
@@nabiltech1366 half way . did you finish
@@MrBemnet1 No bro.The way he teach is very complicated to me.So i decide to learn a new way.When i have a little bit knowledge that i understand,i will try to retake the course so it will be easier than before.What about u?
@@nabiltech1366 I dont some of the concepts right a way.i will check other resources then come back and view it again. I will finish it everything within 2 weeks
Thank you very much, You have been an angel for me. Please upload a video on the theory part of SVM, K-Means or other unsupervised ML. Thanks a lot once again. Hari Om
Sir you are amazing! , you have cleared everything.
Thank you. It was so helpful.
i really love your energy
Hi Krish, Thanks for making such nice videos and excellent explanation. Finally I have found somethingl I was looking for better understanding of deep learning.
Good work as usual krish... Awaiting its implementation 🙏🙏
This man makes ML a cakewalk!
your all videos are very useful ...thanks alot for this good work
Thanks a lot Krish for your best explanation.
Thanks a lot, sir, very good explanation.
Hello Krish.Came to know about the use of random forest in deep learning.Thanks
hats off to you sir,Your explanation is top level, THnak you so much for guiding us...
I am watching your videos from few months and I learned a lot, your channel deserve subscription, I subscribed your channel
very well explained. thankyou
simple and clear explanation
i was alwasy confuse about deep learning beacuse of u i got clarity
Great explanation 👍
Thank you Krish for the video, this is excellent!! One question, drop out will be applied at each epoch, then how does it combine the results from all the epoch?
Great video 👏
Hi Krish, great work, real smooth and informative explanation
Your lectures are superb
I have a doubt -
On every iteration drop out ratio of any particular layer remains same or not? If not then do we take average to multipy with weights for test data ?
Hello @Krish Naik, You mentioned in Video that for test data w should be multiplied by P. Do we need to write a code for that in Model ? Does it happens aromatically?
Extraordinary teaching style step wise.You made all my concepts clear , Can you please add some practical implementation of neural network models in which all these techniques can be used. like dropout, loss function , learning rate , regularization , optimizer in one model implementation..Thanks in advance...
Krish: You are the very best trainer
thank u from Iraq .. Good Job brother
It's really very good lecture series
Great effort Krish! I like your passion. I have a one confusion about drop-out ratio. Why are you using drop-out ratio of 0.5 for input layer ? According to my knowledge that should be higher (i.e 1.0 or 0.9).
Thanks Krish
You teach very well... Gr8 stuff about Data Science in your channel. Thanks Harish!
It's Krish buddy
Thank you for this excellent explanation! could you link the original research thesis you mentioned? (or maybe i'm just not finding in description)
Hello Krishna, first of all thanks you so much for the videos as lot of my queries are getting cleared up by watching your videos. I have a better understanding of Neural Networks now with all the maths behind it. I have one query though for this particular video : What is Batch Normalization in Neural Networks and how does it help in preventing over-fitting problems in a neural network?
Best explained:)
thank you sir.
thank you
Nice Explanation
great!!!
Sir you are great 💖
sir i think your enjoying this teaching ?
your expressions indicating you are enjoying the teaching ...
Sir as we are randomly selecting some features or neurons, then those are being updated according to that set of neurons in that particular FP and BP, so how come the model is going to predict the right answer when all the neurons are activated together for Test data as we have trained the weights of the neurons when there where less number of the activated neuron, like how, the model will sum up all the weights to give the right prediction(with least error).
Amazing Sir
I just have a little query if we keep activating and de-activating neurons while training
doesn't it cause overfitting when testing with all neurons activated at once which were trained in some different combinations during training
In your sketch - did you really drop a couple of inputs out? Is this allowed in dropout approach?
The video explains the concept of dropout layers in deep neural networks, which helps prevent overfitting by randomly deactivating a subset of neurons during training.
Key moments:
00:00 Artificial neural networks with many weights and bias parameters can lead to overfitting issues, dropout regularization helps prevent overfitting by randomly dropping units during training.
-Explanation of overfitting in deep neural networks due to excessive parameters and the need for regularization techniques like dropout.
-Comparison between underfitting in single-layer neural networks and the role of multiple layers in preventing underfitting in deep neural networks.
-Introduction to dropout regularization as a technique to prevent overfitting by randomly dropping units during training, with a reference to the 2014 thesis by Srivastava and Hinton.
03:54 The video discusses the concept of dropout layers in neural networks, where a subset of features or neurons are randomly deactivated during training to prevent overfitting and improve model generalization.
-Explanation of how dropout layers work in neural networks by randomly deactivating a subset of features or neurons during training to improve model generalization.
-Comparison of dropout layers in neural networks to the concept of selecting subsets of features in random forests to create diverse decision trees for better model performance.
07:25 Dropout layer in neural networks randomly deactivates some neurons and activates others during training to prevent overfitting, similar to random forest's feature selection and majority voting. Test data connects all neurons without deactivation or activation, using weights multiplied by dropout probabilities for prediction.
-Comparison of dropout layer with random forest for feature selection and majority voting to prevent overfitting in neural networks.
-Explanation of how test data is handled in dropout layer, connecting all neurons without deactivation or activation, and using weights multiplied by dropout probabilities for prediction.
-Selecting the dropout ratio (p-value) through hyperparameter optimization to prevent overfitting in deep neural networks, with a recommendation for p-value above 0.5.
just a question . during back propogration, for each neuron we get updated weights. Now when we back propogate to starting, and again random starting feature points are chosen, what happens to back propogated weights?
helpful!!
how simply he explained it .
Sir I have a doubt that when the neurons are randomly selected base on p value then for next epochs from which neurons the random selection which will performed activated ones or all of them
i have a question. we have to add different drop layers for different layers or we have to add once for all layer ?
Hi Sir,
I have a doubt.
If we take p=0.5 half of the features which will be deactivated at 1st epoch will be reactivated in 2nd epoch and same goes on for other features in upcoming epochs as well
Please explain
Can there be a better explaination? Simply perfect!!
I have a doubt.
In test data which neurons are not activated we are doing p*w but which neurons are activated what will we doing in that case?
I think during test time we should multiply the weights with keep probability value = (1- dropout rate). Intuitively keep probability means how many % of times we have used that weight or edge or connection to train our NN. please correct me if i am wrong Krish sir.
Hi, Sir. I would like to know in each epoch of training, does dropout have relations to batch_size?
Hi, In this video, when we are going to apply for test data...what will be the weight of deactivated neurons
Sir if we're dropping some input and also hidden layers,
It will not affect our output?
Mean correct predictions
Hi, You did not explain how the exploding problem can be corrected - is it through Same RELU ?
For training data suppose we are ignoring few features and neurons as per the drop out ratio and calculating the weights and with back propagation v r updating the weights. In the second step another set of features and neurons are selected randomly, Now if we are again calculating the new weights that doesn't make sense rights as this will keep on repeating with different random combinations.... Please correct me if I am wrong...Thanks in advance.
so after all of this is done the best set of features are selected for that particular output value i guess
if we apply drop out ratio is there any chance that the features which are selected first time get selected in second time..or new features get selected.
krish..you make my life easier
sir if in train we drop the x2 and x4 features we won't get weights then while testing how those weights(Unknown) get multiplied with drop out ratio. I did not get that ,please explain ..
@Krishna : The features x2 and x4 will be dropped out only in 1st epoch. Once the epoch is completed, again it will select 2 other features as per the drop out ratio. Once this loop gets completed, all the neurons in each layer will have some weight with it.
Super
Sir all weights will be updated as (P*W) while testing data or the P value will be updated as (P*W) ? Please clear this.
Can you please provide link for the Machine Learning playlist?
how can we say that the model is underfitted or overfitted
Can you explain how it is helping to avoid overfitting problem.
Krish i have a doubt. Suppose i have 5 inputs & 5 neurons in my 1st hidden layer. In training time, i have given drop out ratio as 0.5, & due to this suppose 2 inputs & 2 neurons got deactivated. In this case now we have 3 i/p & 3 neuron left, so 9 weights we have to train. But at testing time we have to multiply 'p' value with 25 weights as testing time all i/p & neurons exists. So how to do this?
i think the drop out ratio for other deactivated neurons in the test set would be 0 i guess doesnt make sense though
Hi @Krish,
I got asked in an interview, what if we remove one hidden layer instead of DropOut, wont it be good to remove one Hidden Layer instead of DropOut,
Can you please help me with the Answer.
Please suggest a good reference book for Deep learning.
Hi sir, Amazing explanation..
small doubt..
while multiplying 'p' value with weight 'w' for test data,do we include(add) bias value with input??
We have to include the bias..
How do we determine the value of p
In the next iteration, will the deactivated neurons get activated randomly???
hello sir
just want to ask
during dropout we will drop few neurons and at the time of testing we will connect them all and will update weights of whole network as (W*P)
but what are the weights of drop neurons should we take W=1 for drop neurons
All neurons will get updated with weights as there are multiple forward and backward prorogation. Dropout just prevents a random set of neurons from updating after each forward and backward iteration.
Wouldn't the weights in testing be w(1-p) rather than wp?
Alla👌👌
So when the neurons are reactivated what are their weights?
their weights are the same as before because you didn't update them using backpropagation. You only update the weights corresponding to neurons that are activated at an iteration. So in the next iteration, if we happen to activate the neuron which was not active on the last iteration it's weight will be the same until backpropagation updates it (because that neuron is active now and hence will get updated).
@@akashkewar thanks for the reply after 8 months😃😃😃♥️
@@shiffin_chippe :D "Better late than never". I hope you are doing fine in life and don't give up.
Can you or anyone please provide the thesis or research paper link
How to find dropout layer ratio for my model???