The path from theory to study consists of a thousand decisions, big and small. How and how much do these decisions matter? We discuss a recent crowdsourced meta-study that tried to find out. Fifteen teams of researchers were given 5 different hypotheses and told to design a study to test them, then they ran all the studies and got widely varying results. What are the implications of this study for how we should think about the role of theory in study design? What does it say about the different functions of direct and conceptual replications? Is this evidence of hidden moderators? How predictable were the differences in results, and were they predictable because of differences in the study designers' expertise, biases, or something else? Plus: We answer a letter about getting scooped on a systematic review.
Links:
The Black Goat is hosted by Sanjay Srivastava, Alexa Tullett, and Simine Vazire. Find us on the web at www.theblackgoatpodcast.com, on Twitter at @blackgoatpod, on Facebook at facebook.com/blackgoatpod/, and on instagram at @blackgoatpod. You can email us at letters@theblackgoatpodcast.com. You can subscribe to us on iTunes or Stitcher.
Our theme music is Peak Beak by Doctor Turtle, available on freemusicarchive.org under a Creative Commons noncommercial attribution license. Our logo was created by Jude Weaver.
This is episode 75. It was recorded on January 30, 2020.
Relevant To Your Interests
Testing 1-2-3
Our Most Significant Episode Ever
Don't Trust Me, I'm A Doctor
What Comes Next?
Nobody Goes There Anymore, It's Too Crowded
Being Different
Psychological Science Is Made Out Of People
Because Reasons (with Ellen Evers)
Situation Normal
Come Together
You Can Go Your Own Way (with Katie Corker)
Tech Tales (with Paul Litvak)
The PhD Who Caught the Car
A Blooming, Buzzing Confusion
A Jury of Your Nerdy Peers
Hanging in the Balance
Aboard the Hype Train
Moving along
Academic Kindness
Create your
podcast in
minutes
It is Free
DNA Today: A Genetics Podcast
Museum of the Missing
Strange by Nature Podcast
Sasquatch Chronicles
Hidden Brain