Use experimentation to validate ideas, not score goals

This interview is part of Kameleoon's Expert FAQs series, where we interview leading experts in data-driven CX optimization and experimentation. David Mannheim is the Global VP of CRO at Brainlabs, building a world-class conversion product for digital businesses. Formerly, he founded (and successfully exited) one of the UK’s largest independent conversion optimization consultancies, User Conversion. Also loves Disney. (Too much)

Hi David, thanks for your time! Did growing, and exiting your own business change how you strategize with and advise clients now at Brainlabs?

Absolutely. I have always suggested that the key soft skills within conversion optimization are 3x Cs: Curiosity, creativity and commercialism. At least, when hiring, those are what I continually look for in people.

Curiosity is a virtue. Creativity is cultivated. Commercialism is experienced.

Running your own business doesn’t just teach you to be more commercial, it allows you to experience it. Not just in the day-to-day of running of the business, but selling to others, negotiating with your clients, appreciating that “if you don’t perform, your neck is on the line.” This forces you to focus on what will really bring your client ROI the fastest, promoting value-based prioritization.

Based on your experience, what is the biggest thing stopping teams and companies from scaling up their optimization programs?

Permission. Centralization. Myopic focus on experimentation. Over-indexing revenue gain. There's a few. How long have we got?

It does all start, as many suggest, with the environment. The people around them, above them—believing that optimization is a discipline, and not a mindset, will inhibit it’s potential infinitely.

DOWNLOAD THE REPORT TO SEE HOW HCOS ARE USING DATA INSIGHTS TO IMPROVE PATIENT EXPERIENCE.

In marketing, there is competition between the “numbers” people and the “creatives”—as an animation enthusiast, what would you say experimentation can learn from the creative disciples?

I remember starting CRO, and justifying my love for it because it was a perfect blend of the analytical and the creative. I think my mindset has since matured.

Experimentation is just a methodology to validate. In doing so, it can help us innovate by quickly, and practically, proving or disproving hypotheses. In the problem solving sense, it promotes creativity.

We can also learn a lot about the animation process and apply that to the experimentation methodology, too. I won’t go into the ins and outs of film making, but there is an infinite amount of testing and iteration that happens. I thoroughly recommend reading Creativity Inc by Ed Catmull.

Anyone who follows you on LinkedIn will know that you’re a connoisseur of shopping cart pages. What are the worst mistakes brands still make, and why do you think they make them?

Am I? ? I counted for a research project I have overseen over 800 tests on checkouts alone. That’s quite a lot. Im coming for you, Christian at Baymard!

I’ll get told off for this. I think that brands over-index usability within checkouts. They don’t focus enough on understanding what prevents a user from purchasing—anxiety—and what persuades a user to purchase—motivation.

I firmly believe that checkouts force users to move from system 1 thinking to system 2. A state of aspiration and desire to form and function. Your rational brain kicks in when you fill out forms. You can focus more on what you’re spending, the nuance of receiving your product (shipping), whether and how you should get a discount, etc. It’s not all about speed, it’s about maintaining excitement for the product, reassurance and trust.

They still make these mistakes because of a myriad of reasons: “We don’t know what we don’t know”, poor prioritization, lack of great examples, over-index on a best practice, a belief that there’s a one size fits all solutions, risk of testing on the checkout. Checkout is filled with nuanced scenarios and journeys. There’s a few.

How can experimenters help brands focus more on retention instead of just continuing to pour money into acquisition?

In football, attack wins games and defence wins titles. I feel the analogy is true for acquisition and retention.

Retention is often not seen as sexy as scoring goals. Because it’s revenue realization isn’t as immediate. It’s growth pattern isn’t as evident.

But experimentation is just a methodology to validate. Once that’s realized, or acknowledged, or appreciated, we start to move away from just scoring goals. I.e. Moving away from myopic revenue oriented goals such as “How to convert a user in a single session”. Such is the life of a commercial entity.

And we start to discuss more how the purpose of testing can be to learn by validating hypotheses and assumptions. At that stage, we’ll talk about what matters to customers. How we can not just retain them, but delight them. How our unboxing and packaging influences their brand affinity. How our operations can be experimented to improve the quality of our product. How we can test and introduce new product lines to open new markets.

Again, this all comes down to mindset around the power of experimentation, and moving it away from over-indexed revenue metrics.

Pay it forward? Who do you think deserves to be featured for their work in experimentation?

Unsung heroes who believe in decentralized optimization, fighting it in a centralized world. I’ll opt for Steph Nash.