Batch normalization | What it is and how to implement it

Sdílet
Vložit
  • čas přidán 2. 08. 2024
  • In this video, we will learn about Batch Normalization. Batch Normalization is a secret weapon that has the power to solve many problems at once. It is a great tool to deal with the unstable gradients problem, helps deal with overfitting and might even make your models train faster.
    Get your free speech-to-text API token 👇
    www.assemblyai.com/?...
    We will first go into what batch normalization is and how it works. Later we will talk about why you might want to use it in your projects and some benefits of it. And lastly, we will learn how to apply Batch Learning to your models using Python and Keras. Even though it is fairly simple to apply Batch Normalization using Keras, we will touch upon some details that might need extra care.

Komentáře • 61

  • @ludwigstumpp
    @ludwigstumpp Před 2 lety +33

    Thanks for doing these videos. :)
    As someone who is familiar with Batch Normalization I was personally missing a few important information which is why I add them here for the community:
    - The normalisation is happening over the batch dimension (in contrast to other variants such as layer normalisation where we normalise over the layer dimension), meaning that we normalize the feature over the mini-batch
    - which is why it does not work well for smaller batch sizes (usually 16+)
    - another advantage for the scale and the offset parameter is that it allows the network to undo the BN, meaning that BN can't make your result worse
    - during test time with e.g one sample only we can't compute mean and std since we don't have a batch. This is why we use running statistics of mean and variance calculated during training

  • @AlexKashie
    @AlexKashie Před 11 měsíci +4

    Best explanation of Normalization and Standardization... Thank you

  • @ramiallouch1052
    @ramiallouch1052 Před rokem

    simple and very useful . thank you for this great content

  • @juliusodunuga8911
    @juliusodunuga8911 Před rokem

    Thank you 👍🏻👍🏻👍🏻
    The explanation is superb

  • @celilkaangungor6570
    @celilkaangungor6570 Před 2 měsíci

    Tebrikler , en iyi anlatan kanal !

  • @paulj.murphy7447
    @paulj.murphy7447 Před 10 měsíci

    great work! thank you.

  • @user-fh7yk8nm6s
    @user-fh7yk8nm6s Před rokem

    Very clear explanation!

  • @sajolsajol8393
    @sajolsajol8393 Před rokem

    Thank you....That was awesome...

  • @VritanshKamal
    @VritanshKamal Před 9 měsíci

    Nicely Explained ! I liked the part where we start from definitions.

  • @eyadaiman1559
    @eyadaiman1559 Před 2 lety

    Thank you, Please keep doing this kind of videos, your explanation is simple and clear

  • @karthiklogan9384
    @karthiklogan9384 Před rokem

    Amazing, thanks a lot .

  • @mahdighribi4151
    @mahdighribi4151 Před 2 měsíci

    Best explanation , Thank you

  • @spider853
    @spider853 Před rokem +3

    First thanks for an amazing video, answered many questions. It felt like you predicted my questions during the video and proposed the answers right after!
    I have one question, if we don't normalize the input and use BatchNormalisation, wouldn't it behave completly different?
    Like for example say we feed training images of luminance value of 0-200, but on real world interference or during validation we use some other images that have full scale luminance values 0-255.
    Giving we know the range of our luminance values during modeling, wouldn't be better to use prenormalization, as Batch normalization will behave incorrectly during real world/validation process?
    P.S. To avoid any confusion about the data and why we didn't feed 0-255 before, say we have grayscale images, and we don't know if they're all in range, and what we'll have during validation, basically random split.

  • @mariajoseapolo1989
    @mariajoseapolo1989 Před rokem +1

    Thanks for the info, It was super easy to understand and clear. ( :

  • @donmiguel4848
    @donmiguel4848 Před 4 měsíci

    The hint to omit the bias when using afterwards batchnorm_layers is very good, the information that batch_norm can be used while omitting the learnable scaling and offset would also be helpfull because these functionality is also computational very expensive and not the core feature of the batch_norm.

  • @autodidacte6561
    @autodidacte6561 Před měsícem

    Nice explanation thanks you

  • @nayanparnami8554
    @nayanparnami8554 Před 2 lety +2

    Superb explanation to one of the important interview question..
    Great work.!!👍🏻👍🏻👍🏻
    Thanks for the video.

  • @anshulzade6355
    @anshulzade6355 Před 2 lety

    This is really nice. Please keep up the good work , the world needs it. If possible can you also share the notebook.
    Lots of love from India!

  • @CharlyBraga
    @CharlyBraga Před 9 měsíci

    Thanks

  • @narendrapratapsinghparmar91
    @narendrapratapsinghparmar91 Před 6 měsíci

    THanks

  • @sharadchandakacherla8268

    great!

  • @pauldonzier7230
    @pauldonzier7230 Před 8 dny

    Standardization does not change the overall shape of your distribution, it just translates and scales it to have mean 0 and std 1. It will give a normal distribution if and only if your distribution was already normal. If your distribution was uniform, or poisson, or whatever, it will remain like that

  • @malayali_thaaram
    @malayali_thaaram Před rokem

    Great Explaination!

  • @jeffinkachappilly9708

    Thanks a lot.

  • @mayankanand111
    @mayankanand111 Před 2 lety

    I have watched this video over and over, you explained it very well though I am not a fan of Keras. From scratch, implementation would have been more helpful.

    • @AssemblyAI
      @AssemblyAI  Před 2 lety

      Hey Mayank, thank you! I'm glad to hear it was helpful. -Mısra

  • @nazaninadavoodi3563
    @nazaninadavoodi3563 Před rokem

    perfect explanation 😍

  • @saikalyangonuguntla594

    Your explanation very good keep doing more videos on datascience concepts

  • @tachyon7777
    @tachyon7777 Před 2 lety

    Marvelous.

  • @mehrdadkazemi3969
    @mehrdadkazemi3969 Před 2 lety

    ty

  • @mohammedamirjaved8418
    @mohammedamirjaved8418 Před 2 lety

    Love...

  • @j7m7f
    @j7m7f Před rokem +5

    I got impression that this video is about normalization. There is nothing about batches and what does it mean BATCH normalization

  • @Omsip123
    @Omsip123 Před 6 měsíci

    What I learned from other videos, that all of this is applied across the samples per batch for each weight. Not across all weights. I hope I got it right…

  • @ssnprakash
    @ssnprakash Před 7 měsíci

    worth it

  • @DavidPham86
    @DavidPham86 Před rokem

    nice

  • @DezelDanmark
    @DezelDanmark Před měsícem +1

    Why do we need this activation function?

  • @malikfahadsarwar2281
    @malikfahadsarwar2281 Před rokem

    so if we have three columns age ,weight and height all are in different scale so we don't have to scale them separately instead we can use batch normalization to bring them to same scale

  • @art4eigen93
    @art4eigen93 Před 2 lety

    This is great!... This video needs a much high view count. What is going on @youtube???? you need to work on your ranking algorithm.

    • @AssemblyAI
      @AssemblyAI  Před 2 lety +1

      Hahah that's great to hear that you like the video Aritra!!

    • @AssemblyAI
      @AssemblyAI  Před 2 lety

      Hahah that's great to hear that you like the video Aritra!!

  • @donmiguel4848
    @donmiguel4848 Před 4 měsíci

    Why does she begins with making the difference between normalization and standardization and then @4:30 describe standardization with mean=0 and var=1 under the name "Batch normalization" ,does anyone understand this advance ?

  • @ilkeyigiter
    @ilkeyigiter Před rokem +1

    Are you turkish? We are so curious :)

  • @grownupgaming
    @grownupgaming Před rokem

    Batch normalization works upon all samples in a batch, but only one feature i thought?

    • @grownupgaming
      @grownupgaming Před rokem +1

      2:22 this is two separate features right, number of phones, and amount of money withdrawn from ATM.

  • @Luca-yy4zh
    @Luca-yy4zh Před 2 lety +8

    No one really knows why BN works at the moment. The best intuition we have is that it counteracts the internal covariance shift problem during training.

    • @codematrix
      @codematrix Před rokem +1

      I think you just answered your own question. Its to help keep features and activation values within in a finite range thus avoiding exploding and vanishing gradients. Having said that, though, isn’t that what the sigmoid and htan activation function supposed to do?

    • @bwowekeith4472
      @bwowekeith4472 Před rokem

      @@codematrix well u might end up getting dead Neurons without batch normalization

  • @donmiguel4848
    @donmiguel4848 Před 4 měsíci

    There is a difference between squeezing the data between 0 and 1 and on the other hand pushing the mean to 0 and squeezing the variance to 1. The lady does not seem to understand what the difference is @10:20 ! The normalization layer just can use the statistic of the actual 28x28 image. But the statistic of the other 60000 images are all individual, so they are all normalized based on different metrics in contrast to just squeezing the data between 0 und 1 by dividing by 255. She does not understand what she is doing and misuse the workflow. The network has to work for all input images and so the input layer has to be adjusted to the average statistic of all images if at all. Her initial /255 method is way better for the mnist data than the lazy NormalizationLayer abuse she is advising which is way more computational expensive on top of that.