Andres Glusman is the Cofounder and CEO of DoWhatWorks. Prior to DoWhatWorks he led product and growth at Meetup where he was a pioneer in the lean startup movement. He has been running tests online since the late 1990s and is passionate about human behaviour. We spoke about the power of split testing, when you can do it, and when you can't.
Episode highlights:1. Split tests are a great way to work out how to grow
Otherwise known as A/B tests, split testing involves systematically varying a user experience for different groups of users & then comparing the end result to see how those changes impacted behaviour for better (or worse).
2. Four out of five split tests fail to move the needleThese are terrible odds, but the good news is that terrible odds aren't hard to make a little better. It's important to get as much signal from as many sources as possible upfront so you have the best chance of success.
3. These signals can come from anywhereSplit-testing is a great way to learn, but it's not the only way to learn. Make sure you use a mixture of experiments, surveys, qualitative inputs, feedback and any other data source you can get your hands on. It's all signal.
4. There's only so much juice you can squeeze from a lemonYou need to be careful that you're going overboard. It's possible to test too much & get caught in eternal loops. It comes down to fear of mistakes but, unless you're putting medicine in someone, you can make a mistake.
5. Yes, B2B people can do split-tests tooYou don't have to have a mass-market B2C app to get good tests going. There's scope to experiment in B2B but you need to do your homework, get as much data as you can upfront & make sure you put your chips on the right part of the table.
Contact AndresYou can catch up with Andres on LinkedIn.
Navigating the World of Product Management (with George Nurijanian, Product Owner @ Xero & Founder @ prodmgmt.world)
Fighting Fires in B2B Product Management (with Rich Mironov, Product Consultant & author ”The Art of Product Management”)
Servitising Product Management & Setting Up Product Teams For Success (with Jas Shah, Product Consultant & Founder @ Bitsul)
Curing Product Diseases with a Radical Product Vision (with Radhika Dutt, author ”Radical Product Thinking”)
Driving Organisational Alignment with Survival Metrics (with Adam Thomas, founder @ Approaching One)
Product Leadership Principles for Tumultuous Times (with Giff Constable, former CPO @ Meetup & author ”Talking to Humans”)
Solve Customer Problems Quickly by Joining the No-Code Revolution (with Natalie Furness, founder @ Minimum Viable Stack)
How to Deploy Empathy to Truly Understand User Needs (with Michele Hansen, author ”Deploy Empathy”)
Chinese Startup Culture & Putting the Minimum into MVP (with Carlos Lastres, Creative & Marketing Director @ Kaiyan Medical)
Finding a PM Superpower and Using your Product Sense (with Peter Knudson & Braxton Bragg, co-authors "Product Sense")
Escaping VC Culture & Building Products that Focus on True Lifetime Value (with Rand Fishkin, author "Lost & Founder" & co-founder @ SparkToro)
Taking Data Science from Academia to the Heart of your Product (with Paul Meinshausen, co-founder @ Aampe)
All Aboard! Why You Need To Get Product Onboarding Right (with Ramli John, MD @ ProductLed)
Challenges & Opportunities for Africans Making Products for Africans (with Abisoye Falabi, Senior PM @ TradeDepot)
Getting into the Habit of Continuous Discovery (with Teresa Torres, author "Continuous Discovery Habits")
Platform Product Management & Getting Ready for Launch Day (with John Zilch, Director of PM @ Upland & founder @ Launch Day)
Putting Customers at the Heart of your Product Decisions (with Hubert Palan, founder & CEO @ Productboard)
Applying Agile Principles to Education & Measuring Learning (with Chris Hull, founder & CPO @ Otus)
Data-Driven Product Development & Ethics in AI (with Korbinian Spann, founder & MD @ Insaas)
Avoiding Bulls**t Product Advice & Focusing on the Essential (with Henry Latham, founder @ Prod MBA)
Create your
podcast in
minutes
It is Free
The Commercial Edge: Unleash the Power of People
The emPOWERed Half Hour
NABOR® TALKS
U.S Property Podcast
Aligned Money Show
The Ramsey Show
Planet Money