The path from theory to study consists of a thousand decisions, big and small. How and how much do these decisions matter? We discuss a recent crowdsourced meta-study that tried to find out. Fifteen teams of researchers were given 5 different hypotheses and told to design a study to test them, then they ran all the studies and got widely varying results. What are the implications of this study for how we should think about the role of theory in study design? What does it say about the different functions of direct and conceptual replications? Is this evidence of hidden moderators? How predictable were the differences in results, and were they predictable because of differences in the study designers' expertise, biases, or something else? Plus: We answer a letter about getting scooped on a systematic review.
Links:
The Black Goat is hosted by Sanjay Srivastava, Alexa Tullett, and Simine Vazire. Find us on the web at www.theblackgoatpodcast.com, on Twitter at @blackgoatpod, on Facebook at facebook.com/blackgoatpod/, and on instagram at @blackgoatpod. You can email us at letters@theblackgoatpodcast.com. You can subscribe to us on iTunes or Stitcher.
Our theme music is Peak Beak by Doctor Turtle, available on freemusicarchive.org under a Creative Commons noncommercial attribution license. Our logo was created by Jude Weaver.
This is episode 75. It was recorded on January 30, 2020.
Creativity and Rigor
Welcome To The Big Leagues
The Year 2017 in Review
Whodunnit
Significant Feelings
We Were Never Cool
The Learning Curve
Disagreeing about Science
The Meaning of It All (with Anna Alexandrova)
All Your Idols Have Clay Feet
Big Time
SIPSapalooza
A Psychometrician's Guide to Popularity
The Iron Psychologist (with Jamil Zaki)
'Cause You're There For Me Too
It's So Complicated
Nullius in verba
I Felt Like a Real Scientist
Confessions of a Replicator (with Rich Lucas)
Dancing On Our Own
Create your
podcast in
minutes
It is Free
DNA Today: A Genetics Podcast
Museum of the Missing
Strange by Nature Podcast
Sasquatch Chronicles
Hidden Brain