What is really damaging to the community by papers which are not scientifically honest is that people get excited and carry on over their work, thinking that the direction shows promise, and spend hours (days-weeks-months) of their time trying to replication/implement the paper only to get sub par results, even worse, write their own papers on top of that and lose the track of ideas completely due to not being able to attribute the successes or failures to anything in particular. Like you put precisely, other factors must be kept constant in order for a fair comparison to show what exactly the paper's proposed mechanisms managed to contribute... Thanks for doing these videos and the investigations :D Saves a lot of time for me! (And probably others as well...)
Haha, I honestly do not like making such a series of videos. But I end up doing them anyway: I start reading the papers with great enthusiasm, but end up extremely (negatively) surprised by them. Honestly, I love the SimVLM paper and its contribution. But this also makes me judge it harsher. 😅
What is really damaging to the community by papers which are not scientifically honest is that people get excited and carry on over their work, thinking that the direction shows promise, and spend hours (days-weeks-months) of their time trying to replication/implement the paper only to get sub par results, even worse, write their own papers on top of that and lose the track of ideas completely due to not being able to attribute the successes or failures to anything in particular.
Like you put precisely, other factors must be kept constant in order for a fair comparison to show what exactly the paper's proposed mechanisms managed to contribute...
Thanks for doing these videos and the investigations :D Saves a lot of time for me! (And probably others as well...)
This rant really speaks from my heart! Thanks for the great video!
I love this series of videos: “good results bad paper”. ;-)
Haha, I honestly do not like making such a series of videos. But I end up doing them anyway: I start reading the papers with great enthusiasm, but end up extremely (negatively) surprised by them. Honestly, I love the SimVLM paper and its contribution. But this also makes me judge it harsher. 😅
I enjoyed the coverage. The key indeed is the Data.
This feels like what peer review should be, great series!
This is so informative and great, thank you for your research, madam!
should look into this topic; seems interesting
Your videos are great! Very insightful.
Glad you think so!
Okay, I feel somewhat misled by that paper