Why Apple ITP is a threat for 30% of your traffic - how can you trust your future experimentation insights?
Consumers rightly want their personal data to remain private when they are online, and not be used to unnecessarily track them around the digital world. This has led to increased regulation, such as the GDPR and CCPA (as outlined in this previous blog), as well as driving many tech companies to change how they handle third-party cookies in their browsers.
Chief amongst these is Apple, whose CEO Tim Cook has publicly committed to put protecting customer data privacy at the heart of everything the company does. This commitment extends to its Safari browser, which has included Apple’s Intelligent Tracking Prevention (ITP) technology since 2017.
We spoke to Kameleoon’s Chief Product Officer Frédéric De Todaro to ask him about ITP, its potentially enormous impact on marketers and how they can still effectively run experiments with visitors using Safari.
1 What is Apple ITP in a nutshell?
First released in 2017, Intelligent Tracking Protection is a privacy feature in Safari that now blocks all third-party storage (cookies or Local Storage) from websites. The idea, as the name suggests, is to stop visitors being tracked without their knowledge after landing on a site, increasing user privacy.
2 Why is ITP important to marketers?
Many people might not be worried by this - after all Safari has a very small share of the desktop/PC browser market, with between 8-10% share globally. In fact, that still makes it the third most popular browser worldwide, rising to second place in the US/UK, with 10-12% of the market.
However, when you look at mobile traffic the story is very different. As the default browser on all iOS devices, Safari has a market share of 50-55% of mobile traffic in countries such as the US and UK, making it the most used mobile browser.
Source: StatCounter Global Stats - Browser Market Share
And given the premium nature of Apple’s products often these are exactly the consumers that many brands are most interested in.
3 How does ITP impact marketers?
ITP is important to digital marketers for a number of reasons, including tracking your visitors as they move around the web and identifying between new and returning visitors in your analytics platforms.
What I’m going to focus on is the impact it has on A/B testing and experimentation programs as it can effectively make your results completely unreliable. This is particularly true with the latest March 2020 update to ITP 2.3 which bans the unwanted storage of data in a number of new ways.
How A/B tests work
When you are running an A/B test you target variants of a page/element or content at a particular group. Other visitors see the original page, thus acting as a control. Clearly, if your visitors come back they need to see the same page/element as when they first landed on your site - otherwise your test is null and void.
Therefore, to bucket your visitors in the right group when testing, your A/B testing tool creates a unique identifier the first time a user lands on the site, storing this in a cookie or as a variable in Local Storage. This is then reused on all subsequent page views and visits to identify them and serve them the right content for the experiment.
ITP and A/B tests
This identifier is normally stored in a cookie, which is where problems start with ITP. Initially, the technology automatically limited JavaScript-based cookies (which were most often used to store test group identifiers) to seven days. This meant that a visitor coming to your site on a Monday, and then returning eight days later on the following Tuesday was not recognised as an existing member of an experimental group. They were treated as a completely new visitor, running the risk that they’d see a different variation to when they first visited.
This made your A/B test results completely unreliable, leading to the wrong conclusions being drawn and actioned. Given that the whole point of A/B testing is to make decisions based on data, not intuition, it removes the confidence digital marketers need to experiment. You could make changes to your site based on this data and they could actually have a negative effect, making the user experience worse and impacting conversion rates and revenues. Clearly, that doesn’t help visitors or brands.
4 How can marketers overcome the negative impact of ITP?
Initially providers of A/B testing tools (and web analytics platforms) looked at a range of ways of replacing JavaScript cookies on sites. Local Storage is one of them.
The move to Local Storage
More than six years ago, back in 2014, Kameleoon pioneered a method of tracking visitors within experiments by using cross-domain Local Storage (LS), a standard web technology supported by virtually all browsers. This keeps the visitor identifier in the LS data store on the browser, activating it when a consumer returns to the site and automatically serving them with the right content for any experiment they are bucketed in.
Our implementation was based on a cross-domain model, which delivered consistency across the entire site, including sub-domains. So, for example if your main site was https://www.randomstore.com and your checkout was at https://transaction.randomstore.com it would recognize a user moving from one to the other as being the same, and serve them the same experiment. To learn more about how we have mastered this complex technology, please read this article.
The end of Local Storage
Unfortunately the March ITP 2.3 update introduced the same seven day limit on a number of script-writable storage forms, including Local Storage. So marketers were back to the same problem - if a user returned after more than seven days, they’d potentially see a different variant than on their first visit, making experimentation and testing results again null and void.
Simulating the impact of ITP on a live A/B test
Take the example of a website where 30% of traffic is on Safari. It has 200,000 unique monthly visitors and an average 3% conversion rate.
We ran a simulation on what would happen during an A/B test of two pages (the original version A and variant B). The hypothesis is that variant B performs better than the original version A.
On our chart the blue curve shows the results when affected by ITP, and the orange line is when the impact of ITP is removed using Kameleoon.
Obviously the results are accurate within the first seven days as ITP restrictions have not yet come into play, and during the second week there is not yet enough data to draw conclusions. However, what the simulation shows is that after the second week, due to ITP, the experimentation/testing tool will report that original page A is the winner. This is the exact opposite of the results where the impact of ITP is removed, which show the actual winning result should be version B. Essentially, due to ITP your A/B test produces a completely inaccurate, unreliable result.
The Kameleoon solution for ITP-compliant A/B Testing
While our cross-domain Local Storage solution was extremely successful and compliant with all previous versions of ITP, we understood that Apple might well change how ITP operates. Therefore, we were already working on new innovative ways of ensuring A/B testing data was reliable while being ITP-compliant.
ITP doesn’t set any restrictions on cookies created server-side on the website, as essentially these are used to run the site and deliver it to visitors. However, the more cookies you have running server-side, the greater the impact on performance, as each cookie is sent with all network requests, damaging the user experience. Consequently we've updated our A/B testing and personalization client-side platform and created a snippet server-side that automatically synchronizes one important cookie that contains the unique visitor identifier. This ensures a visitor will always see the same variant of an experiment.
For our personalization customers, Kameleoon will also synchronize the Local Storage if it is empty. If this is the case, which probably means the last visit was over seven days ago, Kameleoon will perform fetch all the data that was present in the Local Storage from our own backend servers. Once this call is over, data will be restored in the exact state it would have been if ITP did not remove it. Normal operations can then resume.
You can read more on the technical side of this in our developer documentation or in this in-depth blog.
The result is the best of both worlds - visitor data is retained for over seven days, without hurting performance. Experimentation data is reliable, allowing you to improve the experience, conversions and revenues. As with all features in our platform implementing the solution is straightforward and seamless, requiring minimum effort from customers.
To find out more about ITP-proofing your A/B testing and ensuring 100% of your experiment data is reliable, get in touch.
