Andres Glusman is the Cofounder and CEO of DoWhatWorks. Prior to DoWhatWorks he led product and growth at Meetup where he was a pioneer in the lean startup movement. He has been running tests online since the late 1990s and is passionate about human behaviour. We spoke about the power of split testing, when you can do it, and when you can't.
Episode highlights:1. Split tests are a great way to work out how to grow
Otherwise known as A/B tests, split testing involves systematically varying a user experience for different groups of users & then comparing the end result to see how those changes impacted behaviour for better (or worse).
2. Four out of five split tests fail to move the needleThese are terrible odds, but the good news is that terrible odds aren't hard to make a little better. It's important to get as much signal from as many sources as possible upfront so you have the best chance of success.
3. These signals can come from anywhereSplit-testing is a great way to learn, but it's not the only way to learn. Make sure you use a mixture of experiments, surveys, qualitative inputs, feedback and any other data source you can get your hands on. It's all signal.
4. There's only so much juice you can squeeze from a lemonYou need to be careful that you're going overboard. It's possible to test too much & get caught in eternal loops. It comes down to fear of mistakes but, unless you're putting medicine in someone, you can make a mistake.
5. Yes, B2B people can do split-tests tooYou don't have to have a mass-market B2C app to get good tests going. There's scope to experiment in B2B but you need to do your homework, get as much data as you can upfront & make sure you put your chips on the right part of the table.
Contact AndresYou can catch up with Andres on LinkedIn.
Debbie Levitt's Hot Take - Democratising our Work means AI is Going to Steal all our Jobs Sooner (with Debbie Levitt, CXO @ DeltaCX and Author "Customers Know You Suck")
Build Better Products at Scale with Product Operations (with Melissa Perri & Denise Tilles, Product Consultants & Co-authors "Product Operations")
Knowing your Customers, Seeking Evidence and Sticking up for Continuous Discovery (with Hope Gurion, Product Leader and Team Coach @ Fearless Product)
Transforming your Organisation to the Product Operating Model (with Marty Cagan, Author "Inspired", "Empowered" and "Transformed")
Applying Product Management Principles to Life (with Miloš Belčević, Author "Build Your Way")
Standing up for User Research... and User Researchers (with Debbie Levitt, CXO @ DeltaCX and Author "Customers Know You Suck")
Building Great Companies through Community-Led Growth (with Lloyed Lobo, Author "From Grassroots to Greatness")
Harnessing Generative AI to Reimagine the Future of Product Management (with Shyvee Shi, Product Lead @ LinkedIn & Author "Reimagined: Building Products with Generative AI")
Enabling Strategic Product Decisions through Product Operations and Portfolio Management (with Becky Flint, CEO of Dragonboat)
Nailing your Brand Marketing by Embracing your Zone of Genius (with Orly Zeewy, Brand Strategy Consultant & Author ”Ready, Launch, Brand”)
Making Sure You Make an Impact through User Research (with Steve Portigal, User Research Consultant & Author ”Interviewing Users”)
Helping Superhero Startup Founders Stay Away from their Kryptonite (with Richard Blundell, Founder @ Vencha & Co-author ”The Go To Market Handbook for B2B SaaS Leaders”)
Making our Product Teams Stronger through Building Communities of Practice (with Petra Wille, Author ”Strong Product People” and ”Strong Product Communities”)
Going Beyond the Dreaded Product Demo and Creating the Perfect Sales Pitch (with April Dunford, Author ”Obviously Awesome” and ”Sales Pitch”)
Nailing your Product/Market Fit Strategy by Focusing on the Mission Critical (with Maja Voje, Growth Strategy Expert & Author ”Go-To-Market Strategist”)
Fearlessly Defeating the Four Horsemen of a Product-Friendly Culture (with Eisha Armstrong, Co-founder @ Vecteris & Author ”Productize” & ”Fearless”)
How to Build Products when the Founder IS the Product (with Saagar Bains, Fractional Product Leader & Former Head of Product @ The Body Coach)
Supporting the Next Generation of Female Product Managers with Women in Product UK (with Namrata Sarmah, Founder @ Women in Product UK & CPO @ INTO)
Paying Off Your Organisation’s Human Debt Through Agility & Psychological Safety (with Duena Blomstrom, Founder & CEO @ People Not Tech)
Moving Beyond Founder-Led Product Development & Setting PMs up for Success (with Jennifer Yang-Wong, VP of Product @ Contrary)
Create your
podcast in
minutes
It is Free
The Commercial Edge: Unleash the Power of People
The emPOWERed Half Hour
Aligned Money Show
Gorse Culture PODcast : The H.R. Detective Agency!
HCI Leadership Revolution
The Ramsey Show
Planet Money