Andres Glusman is the Cofounder and CEO of DoWhatWorks. Prior to DoWhatWorks he led product and growth at Meetup where he was a pioneer in the lean startup movement. He has been running tests online since the late 1990s and is passionate about human behaviour. We spoke about the power of split testing, when you can do it, and when you can't.
Episode highlights:1. Split tests are a great way to work out how to grow
Otherwise known as A/B tests, split testing involves systematically varying a user experience for different groups of users & then comparing the end result to see how those changes impacted behaviour for better (or worse).
2. Four out of five split tests fail to move the needleThese are terrible odds, but the good news is that terrible odds aren't hard to make a little better. It's important to get as much signal from as many sources as possible upfront so you have the best chance of success.
3. These signals can come from anywhereSplit-testing is a great way to learn, but it's not the only way to learn. Make sure you use a mixture of experiments, surveys, qualitative inputs, feedback and any other data source you can get your hands on. It's all signal.
4. There's only so much juice you can squeeze from a lemonYou need to be careful that you're going overboard. It's possible to test too much & get caught in eternal loops. It comes down to fear of mistakes but, unless you're putting medicine in someone, you can make a mistake.
5. Yes, B2B people can do split-tests tooYou don't have to have a mass-market B2C app to get good tests going. There's scope to experiment in B2B but you need to do your homework, get as much data as you can upfront & make sure you put your chips on the right part of the table.
Contact AndresYou can catch up with Andres on LinkedIn.
Building Data Driven Products & Dodging Unsolicited Advice (with Emily Reid, Product Manager @ FCT)
Driving Digital Transformation & Products for Product Managers (with Moshe Mikanovsky, Senior PM @ Procom)
Aligning your Positioning & the Myth of Product Market Fit (with April Dunford, Positioning Consultant & Author "Obviously Awesome")
Design Thinking & Finding the Time for Design Sprints (with Yoav Farbey, Senior Product Manager @ PARK NOW)
Building Your Product Muscles & Developing Strong Product Teams (with Petra Wille, Product Coach and Author "Strong Product People")
Practical Product Management & Demonstrating Product Thinking (with Büşra Coşkuner, Product Consultant & Coach)
The Need for Diverse Perspectives in Product Development (with Candice Poon, Program Manager @ Microsoft)
Effective Hiring for Product Managers & Building Diverse Teams (with Kate Leto, Product Consultant & Author "Hiring Product Managers")
From Management Consultant to Product Manager (with Deepti Tadala, Technical Product Manager @ Synacor)
Building a Culture of Continuous Discovery (with Cindy Alvarez, Author "Lean Customer Development" & Director Customer Research @ GitHub)
Lifelong Learning for Product Managers (with Carlos Gonzalez de Villaumbrosia, Founder & CEO @ Product School)
Product Management Lessons from a PM Turned Teacher Turned PM (with Melanie McKay, Head of Consumer Product @ Rightmove)
Data-driven Product Delivery & Story Points on Everything (with James Rooney, Delivery Manager @ Discovery Inc.)
How to Build an Effective Product Organisation (with Marty Cagan, author of Inspired & Empowered)
Scaling your Product Team with Product Operations (with Christine Itwaru, Senior Director of Product Operations @ Pendo)
How to Crack your Product Management Career (with Jackie Bavaro, co-author of Cracking the PM Interview & Cracking the PM Career)
Redefining Product-Led Growth (with Heather James, Founder @ Product-Led Alliance)
Product Management in Lockdown & the Perils of Hustle Culture (with Jack Stevens, Senior PM @ Publicis Sapient)
Product Marketing & the Importance of Pirate Metrics (with Abby Hehemann, Senior Product Marketing Manager @ GetResponse)
Empathy, Mentorship and the Importance of Diversity & Inclusion (with Ana Kresina, PM @ Redbubble)
Create your
podcast in
minutes
It is Free
The Commercial Edge: Unleash the Power of People
The emPOWERed Half Hour
NABOR® TALKS
U.S Property Podcast
Aligned Money Show
The Ramsey Show
Planet Money