Is web personalization an extension of A/B testing?

October 18, 2015

The emergence of agile and simple solutions in the field of web personalization has made some people question the superiority of one practice over the other: Is it better to personalize than A/B test, or should we privilege A/B testing rather than web personalization?

According to the latest "CRO Report", from the research firm Econsultancy (1), the popularity of conversion optimization practices continues to grow. 70% of European e-commerce merchants have someone dedicated to conversion optimization in their organization, an 11 points increase when compared to 2009, confirming a trend that can be empirically observed: Conversion optimization is a major stake for e-merchants.

It is also considered crucial or important by 89% of the respondents, and there is a strong correlation between those who have a structured approach and those who actually improve their conversion. A less flattering finding: The same respondents are generally not very satisfied with their conversion rate (only 22% say that they are, a figure that is declining!), proving that there is a long road ahead in order to get results matching their expectations.

A/B testing and personalization: Complementary practices

Among all conversion optimization practices, A/B testing has become, in only 5 years, the most popular practice among European e-commerce merchants, while web personalization is the practice which will likely experience the largest growth in the next 3 years.

These practices actually have quite distinct playgrounds. A/B testing will be preferred for improving user experience (navigation flow, highlighting specific graphic elements, ergonomics...) whereas web personalization will predominantly be used to work on the value proposition of our offer (with appropriate content and messages adapted to each of its visitor segments). These two playgrounds may sometimes overlap: For instance, we can customize the user experience of our conversion tunnel (2 or 4 steps, depending on the kind of visitors), or A/B test the relevance of a promotional offer. However, both practices generally have a preferred field of application.

If problems are diagnosed and common to the entire website (unusually high bounce rate on the homepage, "product decrease" problem, confusing conversion tunnel...), there is a good chance that the issue might be related to the ergonomics of your website or to the user experience in the broadest sense: A/B testing will answer these questions with the certainty of having a statistically precise result. (Small) downside: As a compromise, the result would be generalized to an entire population, but if we're dealing with the visibility of graphic elements or user experience in the broadest sense, personalization might not be a very valuable option.

We also recommend a priori targeting options to be used sparingly when A/B testing: It is better to launch the test on the entire population and later identify behavioral differences for each population segment, according to the criteria that matter to you (see our op-ed article "A/B Testing: Segmenting is not Targeting"). If significant differences appear, paying more attention to web personalization could be valuable.

From experimentation to personalization

A/B testing is therefore a natural gateway to web personalization, so much so that the latter practice is sometimes considered as an extension of the former. We can indeed consider that creating a variation for a page (B vs. A) and setting its deviation to 100% on a specific customer segment is a web personalization. But is it the Alpha and the Omega of web personalization?

The observation of the two practices clearly demonstrates that it is not: It didn't take long after the Internet emerged to see the first experiments in web personalization, referred to as "marketing one-to-one" in the early 2000s (with very restrictive solutions at the time, involving lengthy and complex technical projects). The first testing solutions emerged several years later. And it makes perfect sense: As in the real world, we must first define a marketing strategy, our product, our customer segments and then, if necessary, carry out consumer tests to validate the relevance of our offer to our target.

If A/B testing may be considered as a good entry point to web personalization, instead, it makes sense to think in the opposite direction. In addition, a structured approach to web personalization does not necessarily imply a subsequent testing logic. When managing the the sales promotion efforts of a website, we associate an action to a visitor segment based on a predefined marketing decision taken for this entire segment. Web personalizations (whether content-based or promotional activities) are pushed to 100% because they result from a preliminary analysis of visitor data and the overall marketing strategy. It’s not about testing within this segment (the A/B testing logic), but to measure sales within this segment as opposed to the entire population.

Personalization today

According to the same Econsultancy study (1), less than 30% of e-commerce merchants have invested in the field of web personalization. However, the emergence of agile solutions designed specifically for marketing teams is a game-changer. Creating web personalizations (i.e., associating a segment with an action) is now a matter of days or even hours rather than weeks.

Compared to A/B testing, web personalization is much less technical (the main action is not primarily linked to graphical or technical changes to a web page, but rather to reflect on the definition of a segment) and less stringent in the analysis of the results. By using a control population (not exposed to web personalization) to make comparisons, a conversion gain (or loss) is statistically calculated. As long as improvements can be made, web personalization is here to stay. The practice is ultimately much more approachable and natural for marketing teams.

This is certainly one of the reasons why high intensity of use is being observed among "practicing" e-commerce merchants. The price of success may eventually come: By conducting several web personalizations in parallel, we theoretically multiply the number of variations of the website as well as the risk of contradictions between web personalizations (a visitor can belong to two different population segments whom are accorded two different actions), which can quickly increase the feeling of losing control. One should therefore be well-organized with a solution that can identify and solve conflicts, while facilitating the monitoring of various operations on a dedicated dashboard.

What are the most important conversion rate optimization areas?

In a nutshell, the two practices have their own logic and their own benefits. They are perfectly complementary. There is no reason to assert that we must choose one or the other or even necessarily implement both in parallel. If we are to believe the Econsultancy study, A/B testing today is 2.5 times more popular than web personalization, a difference that—according to the same study—is expected to shrink dramatically in the next three years. It makes perfect sense because nothing seems to justify the fact that we should strictly focus on A/B testing in order to improve our conversion rates. It appears that significant web personalization opportunities have not yet been explored. The emergence of agile and dedicated tools now allows us to explore them.

Conversion Rate Optimization Report 2014 Econsultancy, on over 650 British (56%), European (15%) and Asian/Pacific (18%) e-merchants.
In this article, "web personalization" refers to the customization of content and marketing messages based on customer segments. It does not refer to "product recommendation," a complementary practice to web personalization, which is often distinguished by using these two terms.

Topics covered by this article