DINO: Self-Supervised Vision Transformers

[@marioparreno24]

Thanks for the intuitions, faqs and clearly explained topics!

[@soroushmehraban]

Glad you liked it Mario🙂

[@marioparreno24]

​​@@soroushmehraban Just one question. Why is centering only applied to the teacher and sharpening to both the student and the teacher? Could we not apply centering to both?
Maybe if we add both operations to both sides we play a sum 0 game and we have the collapse problem again, I dont know 😅 Maybe we need then artificially create an unbalance

[@soroushmehraban]

@@marioparreno24 From my understanding, sharpening makes the model more confident that this sample belongs to a certain sudo-class (the output label of model that we don't have ground truth).
And we want the student to be kept certain about it and we sharpen it. The less certain the student is, the less certain it is to differentiate samples from different images.
But for images we do both to prevent the mode collapse.
But this is just based on my intuition. Don't quote me on that lol.

[@yiqian22]

As always, thank you very much for the clear explanation - I truly appreciate it! 👏

[@soroushmehraban]

My pleasure!

[@ericsy78]

This is a great video I really appreciate the dedication in each video you post, I learn a lot watching your videos and it has always been helpful to me.

[@soroushmehraban]

Thanks for the feedback! It’s my pleasure

[@AshishJain-iw5md]

Very informative!!!

[@soroushmehraban]

Glad you liked it!

[@alihadimoghadam8931]

Great video, as always 🤘

[@soroushmehraban]

Thanks😃

[@pulakgautam3536]

I love your channel!

[@soroushmehraban]

Thanks for the kind comment! This is really encouraging. Will try my best to come up with more paper reviews in the future.