Your Favorite Research Is (Probably) Wrong: The Replicability Crisis
Vložit
- čas přidán 6. 07. 2024
- Ever heard of the marshmallow experiment? The 10,000 hour rule? How about the Dunning-Kruger effect, the Stanford prison experiment, or willpower fatigue?
These are some of the most well-known pieces of research from the social sciences. And they all share one problem: they're wrong. Or, at least, they're really misunderstood. It's shockingly common for all kinds of holes to be found in research that becomes popular, but popular belief in the theory is rarely updated.
In this video we're exploring social science's Replicability Crisis, and going on a tour of famous research that was later invalidated.
Chapters:
00:00 Intro
01:16 Dunning-Kruger
02:52 Backfire Effect
05:20 Marshmallow Test
07:18 Stanford Prison Experiment
09:04 Willpower Fatigue
09:40 Critical Positivity Ratio
10:37 10,000 Hour Rule
12:27 The Replicability Crisis
Cited Research:
Kruger, J.; Dunning, D. (1999) "Unskilled and Unaware of It: How Difficulties in Recognizing One's Own Incompetence Lead to Inflated Self-Assessments.”
Nuhfer, Edward; Cogan, Christopher; Fleischer, Steven; Gaze, Eric; Wirth, Karl. (2016) "Random Number Simulations Reveal How Random Noise Affects the Measurements and Graphical Portrayals of Self-Assessed Competency.”
Shoda, Y., Mischel, W., & Peake, P. K. (1990). Predicting adolescent cognitive and self-regulatory competencies from preschool delay of gratification: Identifying diagnostic conditions.
Tyler W. Watts, Greg J. Duncan, Haonan Quan. (2018) Revisiting the Marshmallow Test: A Conceptual Replication Investigating Links Between Early Delay of Gratification and Later Outcomes.
B. Nyhan , J. Reifler. (2010) “When Corrections Fail: The Persistence of Political Misperceptions.” Wood, T., Porter, E. (2018) “The Elusive Backfire Effect: Mass Attitudes' Steadfast Factual Adherence.”
Brown NJ, Sokal AD, Friedman HL. (2013). The complex dynamics of wishful thinking: the critical positivity ratio.
Haney, C., Banks, C., & Zimbardo, P. (1973). Study of Prisoners and Guards in a Simulated Prison.
Baumeister, R.F. (2002) Ego Depletion and Self-Control Failure: An Energy Model of the Self's Executive Function.
Carter E.C., Kofler L.M., Forster D.E., McCullough M.E. (2015) A series of meta-analytic tests of the depletion effect: Self-control does not seem to rely on a limited resource.
Brown N.J., Sokal A.D., Friedman H.L. (2013). The complex dynamics of wishful thinking: the critical positivity ratio. Ericsson, A. K. (2008) Deliberate Practice and Acquisition of Expert Performance: A General Overview.
Who Am I: I'm Forrest, the co-author of Resilient (amzn.to/3iXLerD) and host of the Being Well Podcast (apple.co/3rqbfmM). I'm making videos focused on simplifying psychology, mental health, and personal growth.
You can follow me here:
🎤 apple.co/3rqbfmM
🌍 www.forresthanson.com
📸 / f.hanson - Věda a technologie
Love this... It’s such an important topic. The trouble with a digital world is that incorrect information doesn’t go away. People want to feel better about themselves and about their world but that doesn’t mean they want to become researchers or journalists themselves. When the top 5 hits on a google search tell them something, why would they look further?
It's crazy to see you rather than just hear you on the podcast!!! :))
Yes! But really easy just to listen if you wish.
Very interesting - I’ve heard the same thing said about patents - a patent is published and in many cases the discovery cannot be replicated. We need to be cautious and thoughtful.
Even if the 10,000 rule had been valid for violinists, I am baffled why any scientific-thinking person would just assume that this would magically transfer to all other activities.
I'm also baffled as to what people were supposed to do with the information.
I loved this! Such a needed topic, please share more! 😍
I also heard you about the Stanford prison experiment but what I heard was that the participants were volunteers. What kind of person jumps at the chance to be a prison guard? Well, exactly who you'd think
I think it would be worth testing further the idea that power attracts the corrupt as opposed to corrupting those in power
Thank you very much for the time and the effort you put in this. I had a lots of skepticism in regards of Psychology. And when I took the only (Intro to Psychology) course I felt certain that there are many things that did not click with me. Because every time I asked the teacher a question he will hover with some BS without giving me a clear answer. Thanks again for your videos.
Fascinating and thought-provoking information. Thank you for sharing!
Thank you, Forrest, for that important discussion.
As usual, this is well-researched, eloquently presented and beautifully chosen for its relevance and topicality. Thank you very much for putting this together. As a psychologist-in-training, I do wish you had reiterated at the end how many important, correct and interesting studies there are regardless of the very real and troubling replicability crisis.
Yes! The challenge with a video like this is in balancing the primary topic/entertainment portion ("hey, there's a lot of stuff out there that just gets blindly regurgitated that probably isn't accurate") with some of the important caveats ("research is extremely important, and part of what gets us to the stuff that's true is doing a lot of stuff that turns out to be false").
Watching it back I could have definitely reinforced that for a few minutes at the end - it's a long video and I think I got kinda concerned about length by the end.
Crazy. Dunning-Kruger is something I have used to explain why some people are so confident of their opinion but completely lack the knowledge to be able to understand how incorrect they are. It seems to be a common issue at the moment, is there another theory that explains this better?
@@hasslfootA better example are theologists explaining why science is wrong
I'd heard of the correction of the marshmallow test before. Turns out, children are more likely to wait for a greater reward if their experiences have led them to believe they're likely to actually get that reward. Whoda thunk
Well said and done
The ego depletion thing... Anything that takes energy becomes harder to do when you have less energy to draw from. Not surprising
I have learned a great deal from this, thank-you
Glad it was helpful!
are there any famous psychological studies that are known to be highly replicable? or do they not exist?
Absolutely, but some might not be about what you'd expect.
For example, research has consistently show our memory to be fallible. This includes the generation of false memories based on priming. Priming in general has also been shown to have a major impact on our behavior (check out the research on motor priming for example).
Then there's research on things like the "false consensus effect" which has been very sturdy. To simplify we tend to think our views are the consensus view more often than they actually are: psycnet.apa.org/record/1978-03391-001
Then there are foundational works like Tversky and Kahneman on prospect theory that have also been well validated. So yeah, it's definitely out there.
Late to the party, but I would say this: taking into account socio-economic status does not necessarily include the marshmallow experiment. It could simply mean that these traits persist through generations, i.e. socio-economic status is lower because their parents, grandparents, etc. also had time preference issues.
Will power depletion could be glutamate up regulation which does effect decision making
Interesting
I have seen the Dunning Krueger effect on theologists explaining why science is wrong about god so I know is anecdotal but it really does happen, maybe not as often as we with our short perception may think
I think Human Nature is as complicated as the Climate and like the Climate it is not possible to boil it down into a simple jell.
The Social Science and psychology fail to include the entire nature of humans.
I never trust research I trust reality
Research is based on reality but sometimes not enough reality and more information is needed, even if you test 100 children another million may have acted differently, only way to really know is to test 7 billion children