Andres Glusman is the Cofounder and CEO of DoWhatWorks. Prior to DoWhatWorks he led product and growth at Meetup where he was a pioneer in the lean startup movement. He has been running tests online since the late 1990s and is passionate about human behaviour. We spoke about the power of split testing, when you can do it, and when you can't.
Episode highlights:1. Split tests are a great way to work out how to grow
Otherwise known as A/B tests, split testing involves systematically varying a user experience for different groups of users & then comparing the end result to see how those changes impacted behaviour for better (or worse).
2. Four out of five split tests fail to move the needleThese are terrible odds, but the good news is that terrible odds aren't hard to make a little better. It's important to get as much signal from as many sources as possible upfront so you have the best chance of success.
3. These signals can come from anywhereSplit-testing is a great way to learn, but it's not the only way to learn. Make sure you use a mixture of experiments, surveys, qualitative inputs, feedback and any other data source you can get your hands on. It's all signal.
4. There's only so much juice you can squeeze from a lemonYou need to be careful that you're going overboard. It's possible to test too much & get caught in eternal loops. It comes down to fear of mistakes but, unless you're putting medicine in someone, you can make a mistake.
5. Yes, B2B people can do split-tests tooYou don't have to have a mass-market B2C app to get good tests going. There's scope to experiment in B2B but you need to do your homework, get as much data as you can upfront & make sure you put your chips on the right part of the table.
Contact AndresYou can catch up with Andres on LinkedIn.
From High-Frequency Trader to Investor in Good Ideas (with Pooja Parthasarathy, Head of Product @ AbstractOps)
Valentine’s Special! A Love Letter to Problems, not Solutions (with Uri Levine, Founder @ Waze & Author ”Fall in Love with the Problem, not the Solution”)
Adam & Jason Talk All About Product Strategy (with Adam Thomas & Me, Two Product Management Talking Heads)
The Power of Storytelling for Cross-Functional Alignment (with Ronke Majekodunmi, Director of Product Management @ Promevo)
OKRs: The Gateway Drug to Agility & Good Product Management (with Jeff Gothelf, Product Management Consultant & Co-author ”Lean UX” )
Getting the GIST of Evidence-Guided Product Development (with Itamar Gilad, Product Management coach, speaker and author)
Practice Makes Perfect: Embracing the Messy Reality of Product Management (with Matt LeMay, Product Management Consultant & Author ”Product Management in Practice”)
Going Global! When and How to Take your Product International (with Chui Chui Tan, International Growth Adviser & Director @ Beyō Global)
Helping Tech Teams Get Closer to ”The Business” with Effective Conversations (with Douglas Squirrel, Founder @ Squirrel Squared & Co-Author ”Agile Conversations”)
The Five Dysfunctions of Product Management Teams (with Saeed Khan, Founder @ Transformation Labs)
OKIP LIVE! Is Product/Market Fit Really Dead, or Just Resting? (with Andrea Saez & Dave Martin, Right To Left)
Learning the Bedrock Skills of Innovators & Entrepreneurs (with Bob Moesta, Co-creator of the Jobs to be Done Framework & Author ”Learning to Build”)
Using The Power of Community to Help you Thrive in a Generalist World (with Milly Tamati, Founder @ Generalist World)
Making Sure you REALLY Know your Customers and Pulling out of Growth Stalls (with Adrienne Barnes, Founder @ Best Buyer Persona)
Team Change is Inevitable - What’s Important is How we Respond to It (with Heidi Helfand, Author ”Dynamic Reteaming”)
The Trouble with Product Management Onboarding and How to Get it Right (with Mirela Mus, Founder & CPO @ Product People)
Using Tech to Change Culture and Un-f**king the Design Industry (with Mackenzie Daisley, CEO & Founder at Brieft)
Getting the Narrative Right and Taking Web3 Products Mainstream (with Rachael Sacks, Crypto Storyteller & Founder @ Narrativ3)
The Importance of Messaging & ROARing Your Way to Product/Message Fit (with Diane Wiredu, Founder & Messaging Strategist @ Lion Words)
Supporting Mental Health & Wellbeing for Product Managers (with Jacquelyn Guderley, Product Manager @ OVO & Co-Founder @ Product Mind)
Create your
podcast in
minutes
It is Free
The Commercial Edge: Unleash the Power of People
The emPOWERed Half Hour
NABOR® TALKS
U.S Property Podcast
Aligned Money Show
The Ramsey Show
Planet Money