Eventhough I still find it hard to really be able to explain this in my own words, this definitely helped me with my first understanding about resnets. Thank you for sharing this!
Ive been watching other youtube videos for about an hour and as much as I understood the mathematical and the research problems, they couldn't clearly explain intuitively why it is useful aside from, oh, it just use skip connections to give higher accuracy at deeper depth (ignoring what is the point of identity mapping and why we need skip connection and so forth). Your video blew me away and gave me that eureka moment. Thank you, sir.
I don't understand the different function classes (generic and nested), the universal approx. th states that even a single-layer nn can learn any function.
So... not much difference in terms of what you need to use for each then. I was hoping that resnext architecture would be more efficient and require less computational resources to achieve similar or higher accuracy results.
I am sceptical of this explanation, because convolutional layers and fully connected layer can both learn the identity function, so a network consisting of those falls in the nested function class category already without the residual trick.
Actually, learning the identity function might not be quite so easy due to the subsequent nonlinearlity. They can sort-of kind-of learn it but not easily. On the other hand, if you have an identity function directly available, it's a much better inductive bias.
How can a lecturer explain things but his words were never definite? Even the first figure is wrong, deeper networks can represent the shallower networks just fine. They just have different inductive bias
Hey Alex, please don't stop uploading. It is really hard for us engineers to understand the cutting edge. But this video made things so easy. Thanks
Eventhough I still find it hard to really be able to explain this in my own words, this definitely helped me with my first understanding about resnets. Thank you for sharing this!
Ive been watching other youtube videos for about an hour and as much as I understood the mathematical and the research problems, they couldn't clearly explain intuitively why it is useful aside from, oh, it just use skip connections to give higher accuracy at deeper depth (ignoring what is the point of identity mapping and why we need skip connection and so forth). Your video blew me away and gave me that eureka moment. Thank you, sir.
Thanks for sharing Alex
I don't understand the different function classes (generic and nested), the universal approx. th states that even a single-layer nn can learn any function.
sir your work is appreciated
So... not much difference in terms of what you need to use for each then. I was hoping that resnext architecture would be more efficient and require less computational resources to achieve similar or higher accuracy results.
I am sceptical of this explanation, because convolutional layers and fully connected layer can both learn the identity function, so a network consisting of those falls in the nested function class category already without the residual trick.
Actually, learning the identity function might not be quite so easy due to the subsequent nonlinearlity. They can sort-of kind-of learn it but not easily. On the other hand, if you have an identity function directly available, it's a much better inductive bias.
Awesome
How can a lecturer explain things but his words were never definite?
Even the first figure is wrong, deeper networks can represent the shallower networks just fine. They just have different inductive bias