EP34: Don't Freak Out About Quantitative Results

When you're in the early stages of building a business, you need to move quickly. But when you're testing growth channels, there is a difference between moving quickly and freaking out.

In this episode of the InDemand podcast, host Asia Orangio, founder of DemandMaven, uncovers the pitfalls that teams can fall into when reacting to quantitative results.

She covers the mistakes that founders make when reacting to experiments and how to set yourself up for success when running tests and reacting to the results.

TL;DL

0:22 - Don't freak out about bad numbers. Early technical founders have a tendency to overreact to test data from marketing and growth campaigns. But in growth you're working with humans, not code, and you can't always trust immediate results.

4:35 - A one to two week experiment with low spend on a growth channel likely won't show you a clear result. If you react to it you might be missing a big opportunity.

8:00 - A company Asia worked with was testing out a video demo option. After one week and getting two requests for the video the team was resigned to the fact that it wasn't working, but after waiting it turned out that having the video was actually increasing the number of prospects entering the pipeline.

12:45 - You always have to remember that quantitative outputs can tell you 'what', but can't tell you 'why'. Before you react, you have to pause and dig deeper to understand what is driving the numbers.

20:07 - When you're running a test, the sample size matters. You need a large enough volume of results to be able to trust the results. There is a natural variance in the results you get and if you end an experiment early you might just be seeing variance.