Highlighting the link between digital experimentation and growth

What are the financial benefits of digital experimentation? To find out, global experimentation consultancy GO Group Digital benchmarked brands across the world, looking at the link between growth and the use of experimentation. We interviewed Chris Goward of Conversion.com (ex-Widerfunnel) about the findings and what they mean for brands.

1 What were the key findings of the study?

Well, we know that organizations that create a healthy experimentation culture grow at least eight times faster than global GDP. In many ways this isn’t surprising - to meet changing customer needs and outpace greater competition brands need to be constantly testing and learning if they want to succeed. Our latest research report set to find out the differences between fast-growth and slower growth companies.

What we found is that 89% of businesses we surveyed across the world said that experimentation is important to transforming in the age of the digital consumer. Yet a large number of brands admitted that they haven’t yet created a dedicated digital experimentation programme:

• 13% of German respondents
• 25% of companies in Australia and New Zealand
• 27% of brands in the United Kingdom and Ireland
• 32% of North American brands surveyed

This gap will mean they don’t reap the benefits of experimentation and slip behind their fast-growth rivals.

2 How was the research conducted?

Previously we’d analyzed what makes up effective experimentation practice. We built on that for this study, carrying out qualitative research with 620 individuals at companies that had the capacity to embrace digital experimentation. We asked them a series of questions around their growth rates and experimentation practice, and used this to make a concrete link between the pillars of a successful experimentation culture and performance. We’ve termed this the experimentation program health score.

3 What are the components of experimentation success?

We’ve identified four key pillars that underpin high performance and high growth:

1. Linking experimentation to business goals
2. Customer-centricity
3. A culture of trust
4. Executive buy-in

We rated companies on a scale of 0-5 for each of these pillars, and used this to create their experimentation health score. At the same time we asked companies to provide information on their growth rates - Fast (above 17.5%), Moderate (1% to 17.5%) and Slow (0% or below). We correlated the two to show the link between experimentation success and performance.

4 Pillar 1: Business Goal

Experimentation goals need to be linked to business objectives - essentially people need to see why experimentation supports and works with the company’s overall strategy if they are to support the program. Fast-growth companies were twice as likely to have agreed an overall evaluation criterion (OEC) with the executive team and to then use this to measure experimentation effectiveness across the organization. This was often a blend of metrics - for example revenue/increase in customer numbers and customer satisfaction.

Showing the impact of having a clear business goal, 90% of fast-growth companies said their core daily activities were closely linked to a KPI, against 48% of slow-growth brands. By measuring the program against the right metrics, it drives greater results and unlocks further investment.

5 Pillar 2: Customer-centricity

Being truly customer-centric means using multiple methods to understand what customers want and need and focusing on data to drive experimentation decisions. The study found that slow-growth companies had a much narrower range of methods for collecting insight from their customers - they were twice as likely to use a single tactic than their fast-growth peers.

Using multiple methods allows brands to triangulate their customer insight data - for example A/B testing across the wider customer base and then backing this up with deeper quantitative data from focus groups or user testing with a smaller group of customers. 50% of fast-growth companies reported using data to identify areas of optimization, double the figure for slow-growth brands.

6 Pillar 3: A culture of trust

In order to experiment, teams have to embrace the possibility of failure - and to see it as a learning experience, rather than a negative that will impact their careers. Otherwise, teams will not explore new opportunities and take risks that lead to digital innovation. A culture of trust will help individuals thrive - they will feel accountable, that their work has purpose and they are empowered to act. It drives agility and faster decision making - 90% of respondents from fast-growth companies in English speaking countries reported ‘fast’ or ‘very fast’ decision making, compared to 30% of slow-growth ones.

Unsurprisingly this pillar saw the biggest gap in scores between fast and slow-growth brands. This is because building a culture of trust, particularly in large, traditional organizations can be hard as it means changing the status quo and requires buy-in from across the company.

7 Pillar 4: Executive buy-in

Fast-growth companies were four times more likely to have their experimentation programs driven by senior management, compared to slow-growth brands. There was a direct correlation between the seniority of the executive responsible for experimentation and the likelihood of the company reporting fast-growth.

Executive buy-in not only fosters accountability and the psychological safety needed to build an experimentation culture, but it also helps break down silos and drive company-wide results. Essentially taking a top-down approach delivers better benefits to bottom-up, where companies focus experimentation in specific teams or areas./p>

8 Were there any surprises in the research?

The results highlight a lot of differences between regions, which indicated that companies are approaching experimentation in multiple ways. For example, while the UK and Ireland scored highly in all four pillars, the lowest scoring company was 4x below the highest when it came to performance. This shows significant opportunities to improve, even in a relatively mature market.

Drilling into the results for my own area, North America, we can see that brands take a tech first approach, where companies tend to go out and buy the latest solution. There’s almost a fear of missing out, which leads to adoption of tools ahead of setting business strategy. Other countries do this the other way round - objectives first and then relevant tech second. This ‘tech addiction’ can lead to expectations that just buying a solution will provide a silver bullet to drive success.

I think a big reason behind this approach is that there is so much data in the US that companies can become over-reliant on it. They need to look at other ways of gaining customer insight alongside simply analyzing data, such as sitting down with customers. A good example is personalization - the benefits of the technology are all around creating relevance for customers, but you need to take a strategic not a tactical approach.

9 What’s your advice for brands?

I’d say there are three topline tips I’d give to brands, and they all stem from the fact that there are five components in a successful experimentation strategy - process, accountability, culture, expertise and technology. You need all five to be working together, otherwise you’ll have a bottleneck that limits effectiveness and decreases insight velocity.

So the first point is to look at where your problems are and fix those first if you want to drive improvements. Supercharging one area at the expense of others won’t deliver the benefits you expect as the bottleneck will remain.

The second piece of advice is about tracking the right metrics. If you run lots of experiments but then measure the wrong things, such as bounce rates, you’ll get completely invalid results - these will mean you go off in the wrong direction when it comes to decision-making.

Thirdly, do get the right tools and technology in place. Ensure they are integrated with your strategy and are set up properly and you have the right support from the vendor in place.

For a more detailed analysis of the research download the full study from the GO Group Digital website here.