How Does Batch Normalization Work

Vanishing/Exploding Gradients are two of the main problems we face when building neural networks. Before jumping into trying out fixes, it is important to understand what they mean, why they happen and what problems they cause for our neural networks. In this video, we will learn what it means for gradients to vanish or explode and we will take a quick look at what techniques there are in order to deal with vanishing or exploding gradients.
Previous lesson: • How to Choose an Activ...
Next lesson: • Gradient Clipping and ...
📙 Here is a lesson notes booklet that summarizes everything you learn in this course in diagrams and visualizations. You can get it here 👉 misraturp.gumroad.com/l/fdl
👩‍💻 You can get access to all the code I develop in this course here: github.com/misraturp/Deep-lea...
❓To get the most out of the course, don't forget to answer the end of module questions:
fishy-dessert-4fc.notion.site...
👉 You can find the answers here:
fishy-dessert-4fc.notion.site...
RESOURCES:
🏃‍♀️ Data Science Kick-starter mini-course: www.misraturp.com/courses/dat...
🐼 Pandas cheat sheet: misraturp.gumroad.com/l/pandascs
📥 Streamlit template (updated in 2023, now for $5): misraturp.gumroad.com/l/stemp
📝 NNs hyperparameters cheat sheet: www.misraturp.com/nn-hyperpar...
📙 Fundamentals of Deep Learning in 25 pages: misraturp.gumroad.com/l/fdl
COURSES:
👩‍💻 Hands-on Data Science: Complete your first portfolio project: www.misraturp.com/hods
🌎 Website - misraturp.com/
🐥 Twitter - / misraturp

[@pra1699]

This topic is very complex , might require a rewatch for me. You are very good in teaching.

[@misraturp]

Thank you! Good to hear you liked it :)

[@BobbyWicked]

Very nice! Re: incorrect calculations, there's a typo at 5:40 on the right side version of x hat. I believe you meant 46 rather than 46^2?

[@bay-bicerdover]

well spotted

[@aadilminhaz1637]

Perfect explanation in the most simplest way. 👏

[@GregThatcher]

Thanks!

[@bay-bicerdover]

Sanatina vâkif bir kadinsiniz, batch normallestirme katmaniyla ilgili acik ara en aciklayici video olmus

[@massoudkadivar8758]

Best teacher ever, thanks

[@misraturp]

Wow, thanks!

[@nguyenhaidung8833]

Hi Misra, for your previous example with Mnist, you divided the input values with 255, is that batch normalization for the input layer ?

[@bay-bicerdover]

see definition of normalization in 2:10

@@bay-bicerdover 1:30

[@AbdallahBoukouffallah]

You are so pretty i can't stop watching you videos