Andres Glusman is the Cofounder and CEO of DoWhatWorks. Prior to DoWhatWorks he led product and growth at Meetup where he was a pioneer in the lean startup movement. He has been running tests online since the late 1990s and is passionate about human behaviour. We spoke about the power of split testing, when you can do it, and when you can't.
Episode highlights:1. Split tests are a great way to work out how to grow
Otherwise known as A/B tests, split testing involves systematically varying a user experience for different groups of users & then comparing the end result to see how those changes impacted behaviour for better (or worse).
2. Four out of five split tests fail to move the needleThese are terrible odds, but the good news is that terrible odds aren't hard to make a little better. It's important to get as much signal from as many sources as possible upfront so you have the best chance of success.
3. These signals can come from anywhereSplit-testing is a great way to learn, but it's not the only way to learn. Make sure you use a mixture of experiments, surveys, qualitative inputs, feedback and any other data source you can get your hands on. It's all signal.
4. There's only so much juice you can squeeze from a lemonYou need to be careful that you're going overboard. It's possible to test too much & get caught in eternal loops. It comes down to fear of mistakes but, unless you're putting medicine in someone, you can make a mistake.
5. Yes, B2B people can do split-tests tooYou don't have to have a mass-market B2C app to get good tests going. There's scope to experiment in B2B but you need to do your homework, get as much data as you can upfront & make sure you put your chips on the right part of the table.
Contact AndresYou can catch up with Andres on LinkedIn.
Inside the Mind the Product Acquisition - Why Now & What’s Next? (with James Mayes, Evangelist @ Pendo & Co-Founder @ Mind the Product)
Developer Relations & Product Management - Friends or Foes? (with Tessa Kriesel, Head of Platform DevRel @ Snap & Wesley Faulkner, Head of Community @ SingleStore)
Using Solution Tests to Make Sure You’re Building Products Users Want (with Jim Morris, Founder @ Product Discovery Group)
Building Life-Centred Products with Collaborative Product Discovery (with Sophia Höfling, Co-founder & Head of Product @ Saiga)
Using the Power of Community to Grow Your Startup (with Anna Maste, Founder @ Boondockers Welcome & Subscribe Sense)
Pragmatic Digital Transformation in Traditional Industries (with Dan Chapman, Director, Product Line Leader @ Merck)
Understanding & Interrupting Cognitive Biases in Product Design (with David Dylan Thomas, Author ”Design for Cognitive Bias”)
Building Disruptive Products by Harnessing Power Users (with Moustapha Seck, Founder @ Fluid)
We’re All Responsible For Accessible Product Design (with Holly Schroeder, Senior UX Researcher & Accessibility Advocate)
Aligning Teams with Data & Finding Your Second Product/Market Fit (with Matt Arbesfeld, CEO & Co-Founder @ LogRocket)
Supporting Product Managers & Standing Up for the Value of Product Management (with Princess Akari, founder @ PeopleInProduct)
Adventures in Product Management (with Dan Olsen, author ”The Lean Product Playbook”)
Getting On Top Of Your User Stories & Requirements (with Richard Awe, founder @ Requstory)
Escaping the Build Trap with Product Operations and Strong CPOs (with Melissa Perri, author ”Escaping the Build Trap”)
Climbing the Product Management Career Ladder ... and Working Out if it’s For You (with Katerina Suchkova, Founder @ Ahead of Product)
Build High Growth Products by Following the Product Science Success Path (with Holly Hester-Reilly, founder @ H2R Product Science)
The Importance of Getting Your Product Launches Right (with Derek Osgood, Founder & CEO @ Ignition)
Standing Up for Diversity & Inclusion When No One Looks Like You (with Merina Khanom, Product Manager @ BBC iPlayer)
Getting Strategic to Land Your First Product Management Job (with Crystal Parker, Product Manager & Entry Level Career Coach)
Test Business Ideas Ethically by Experimenting With, Not On, Customers (with David Bland, Co-author ”Testing Business Ideas”)
Create your
podcast in
minutes
It is Free
The Commercial Edge: Unleash the Power of People
The emPOWERed Half Hour
NABOR® TALKS
U.S Property Podcast
Aligned Money Show
The Ramsey Show
Planet Money