Andres Glusman is the Cofounder and CEO of DoWhatWorks. Prior to DoWhatWorks he led product and growth at Meetup where he was a pioneer in the lean startup movement. He has been running tests online since the late 1990s and is passionate about human behaviour. We spoke about the power of split testing, when you can do it, and when you can't.
Episode highlights:1. Split tests are a great way to work out how to grow
Otherwise known as A/B tests, split testing involves systematically varying a user experience for different groups of users & then comparing the end result to see how those changes impacted behaviour for better (or worse).
2. Four out of five split tests fail to move the needleThese are terrible odds, but the good news is that terrible odds aren't hard to make a little better. It's important to get as much signal from as many sources as possible upfront so you have the best chance of success.
3. These signals can come from anywhereSplit-testing is a great way to learn, but it's not the only way to learn. Make sure you use a mixture of experiments, surveys, qualitative inputs, feedback and any other data source you can get your hands on. It's all signal.
4. There's only so much juice you can squeeze from a lemonYou need to be careful that you're going overboard. It's possible to test too much & get caught in eternal loops. It comes down to fear of mistakes but, unless you're putting medicine in someone, you can make a mistake.
5. Yes, B2B people can do split-tests tooYou don't have to have a mass-market B2C app to get good tests going. There's scope to experiment in B2B but you need to do your homework, get as much data as you can upfront & make sure you put your chips on the right part of the table.
Contact AndresYou can catch up with Andres on LinkedIn.
Going Beyond Idealistic Book Principles & The Myth of Unicorn Product Managers (with Emily Tate, Managing Director @ Mind the Product)
Product Management in Africa & Dreams of an African Silicon Valley (with Layo Ogunbanwo, Founder @ Practical Product)
Build What Matters with Vision-Led Product Management (with Rajesh Nerlikar, CEO @ Prodify & author "Build What Matters")
Retail Product Management in a Global Pandemic (with Rhiana Matthew, Senior PM @ Publicis Sapient)
Reducing Waste by Only Spending Time on Really Good Ideas (with Julia Shalet, author "The Really Good Idea Test")
Maintaining Radical Focus and Staying on Strategy with OKRs (with Christina Wodtke, Author "Radical Focus" & "The Team That Managed Itself")
The Philosophy of Product Design (with Udhaya Kumar Padmanabhan, Global Strategic Design Director at Designit)
From Engineer to Product Leader & Getting Away from the "How" (with Rekha Venkatakrishnan, Senior Manager, Group PM @ Walmart)
Data-Driven Decisions & the Product Engagement Score (with Stephanie Tanzar, Director of PM @ Pendo)
Is this Seriously Game Over for Scrum? (with David Pereira, Head of PM @ Virtual Identity, Editor @ Serious Scrum)
From Physiotherapist to Product Manager (with Samuel Ogunkoya, Product Manager @ ProducteevTech)
Product-Led Growth - Game Over for the Sales Team? (with Wes Bush, author "Product-Led Growth" & founder @ ProductLed)
Purposeful Product Transformation & Setting Leaders Up For Success (with Stephanie Leue, CPO @ MindEx & Product Leadership Coach)
Defeating Bias, Prejudice and Bullying in the Workplace (with Kim Scott, author "Radical Candor" & "Just Work")
Sustainable Entrepreneurship & Escaping Hustle Culture (with Polina Marchenko, Founder @ SIDE PRJCT)
Plan your Product Journey via Strategy, Roadmaps and OKRs (with Nacho Bassino, CPO @ Best Day Travel & author "Product Direction")
Surviving a Lack of Product Thinking & Riding the Product Maturity Curve (with Nis Frome, Co-founder & VP Product @ Feedback Loop)
Treating your Career as a Product & Making a Wicked Impact as a Product Leader (with Gibson Biddle, former VP Product @ Netflix)
Growth Marketing & Reshaping the World of eCommerce (with Nichole DeMeré, CMO @ Reeview)
Cleaning Up After Professional Services & Becoming Truly Product Led (with Paul Ortchanian, Founder @ Bain Public)
Create your
podcast in
minutes
It is Free
The Commercial Edge: Unleash the Power of People
The emPOWERed Half Hour
NABOR® TALKS
U.S Property Podcast
Aligned Money Show
The Ramsey Show
Planet Money