Metascience Matters

300+ retractions, image manipulation, and why science should be boring | Metascience Matters #3

Mu Yang is a behavioral neuroscientist at Columbia University, and a scientific sleuth responsible for more than 300 retractions. She led an effort that discovered more than 130 fraudulent papers in the publication record of Eliezer Masliah, former head of the Division of Neuroscience at the National Institute on Aging at the National Institutes of Health. Her sleuthing work has been documented in the book "Doctored" by Charles Piller, Science Magazine, and other outlets, and is unpaid.CONTACT RANDY:metasciencematters@gmail.comEPISODE LINKS:Books: Doctored by Charles Piller: https://www.simonandschuster.com/books/Doctored/Charles-Piller/9781668031254Unreliable by Csaba Szabo: https://cup.columbia.edu/book/unreliable/9780231216241/Scientific integrity blogs: Dorothy Bishop: https://deevybee.blogspot.com/Leonid Schneider: https://forbetterscience.com/Podcasts for critical thinkingPlain English by Derek Thompson: https://www.theringer.com/podcasts/plain-english-with-derek-thompsonThe Gray Area by Sean Illing: https://www.vox.com/the-gray-areaThe Ezra Klein show (NYT): https://www.nytimes.com/column/ezra-klein-podcastOUTLINE:0:00 - Introduction2:58 - Mu's origin story4:35 - Moving to Columbia6:15 - How Mu became a sleuth8:13 - Reporting her first case13:09 - Red flags Mu looks for in papers17:30 - Reductionism in behavioral neuroscience18:04 - Standardization vs. Generalizability19:58 - Data sharing standards across fields21:09 - Difficulties of reporting irregularities in papers, university incentives23:54 - Allocating time between images, numerical, other kinds of data24:37 - How she searches through papers25:45 - Examining the chemistry literature31:10 - Types of misconduct vary by field, risks of reporting35:43 - The case of Eliezer Masliah40:31 - Why demonizing individual scientists isn't productive; the system isn't working56:59 - Academic incentives for positive data1:07:31 - Hard to publish null data; "unhealthy codependence" between academia and publishing1:13:08 - Changing incentives1:21:42 - Are we even making a dent in the scale of scientific misconduct? 1:27:35 - Mu's toolkit1:29:38 - Mu does this work because it's fun!1:34:38 - Protecting students; telling them that null data is ok1:37:52 - Evaluating researchers 1:43:15 - Is peer review still relevant?1:51:38 - How much better could science be?1:55:14 - What will science look like in a century?1:58:13 - Advice and resources for listeners2:00:54 - Online presence2:01:35 - Outro

Esketamine for depression, registered reports, and alcohol use disorder | Metascience Matters #2

Florian Naudet is a Professor of Therapeutics at Rennes University. As a metascientist and psychiatrist, his research interests lie in developing and evaluating methodological solutions to treatment assessment, primarily but not exclusively for mental health conditions. His work has also made inroads to quantifying and understanding research waste and the prevalence of substandard data-sharing practices.CONTACT RANDY:Feedback: metasciencematters@gmail.comEPISODE LINKS:Efficacy and safety of esketamine for “treatment resistant depression”: registered report for a systematic review with an individual patient data meta-analysis of randomized, double-blind, placebo-controlled trialshttps://link.springer.com/article/10....Vibration of effects from diverse inclusion/exclusion criteria and analytical choices: 9216 different ways to perform an indirect comparison meta-analysishttps://link.springer.com/article/10....Pharmageddon (book) https://davidhealy.org/pharmageddon-i...A manifesto for reproducible sciencehttps://www.nature.com/articles/s4156...Using reporting guidelines to improve the reproducibility of cooking Christmas tree meringues: the “People tasting trees” cluster-randomised controlled trialhttps://pmc.ncbi.nlm.nih.gov/articles...OUTLINE:0:00 - Introduction0:36 - Florian's origin story as a psychiatrist and metascientist4:46 - How Florian became interested in metascience11:55 - Systematic review of clinical trials of Esketamine for depression28:45 - Publishing this study as a registered report40:49 - Vibration-of-effects analysis of double-blind randomized controlled trials assessing nalmefene and/or naltrexone for treating alcohol use disorders59:58 - Advice for listeners interested in pursuing research like Florian's 1:03:38 - Outro

Exposomics, Vibration-of-Effects, and the Future of AI in Health | Metascience Matters #1

Chirag Patel is an Associate Professor of Biomedical Informatics at Harvard Medical School, renowned for his expertise in using computational methods to understand human health and disease from high-throughput data streams. He specializes in understanding the role in the intersection of genetics and environmental exposures (the exposome) in human health, as well as various disease areas such as diabetes, cardiovascular disease, and Alzheimer’s disease.CONTACT RANDY:Feedback: metasciencematters@gmail.comEPISODE LINKS:Chirag’s Lab: https://www.chiragjpgroup.org/TEDx talk on the exposome:    • Exposome: decoding human health and diseas...  OUTLINE:0:00 - Introduction0:37 - How do you decide what to work on?1:57 - Where does metascience fit into your work?3:57 - Vibration-of-effects analysis6:41 - How does VoE change how we see existing scientific work?9:22 - The biggest challenges in the replication crisis within biomedical informatics11:48 - Environment-wide association study of Type 2 diabetes15:04 - The exposome17:30 - Policy vs. precision medicine applications of the exposome20:09 - The patient exposome22:38 - HBA1C for diabetes as a bridge to the exposome23:54 - Broader metascientific issues of the exposome25:01 - The effects of extreme weather events on human health29:35 - AI for biomedical informatics, the exposome, metascience31:19 - Advice for listeners interested in pursuing research like Chirag’s 32:53 - Outro