Boost your Klaviyo email campaigns with A/B testing

Email marketing platforms like Klaviyo come equipped with capabilities to A/B test emails to optimize open rates and CTR.

And as your organization’s email marketing programs gets more advanced, you can further segment your list using more data from other areas of your company. This can help you further optimize messaging for your customers.

However, with experimentation tools like Kameleoon, you can now segment your lists based on the customers’ actions on your website. You can take user data from website A/B tests and apply it to your email marketing campaigns.

Benefits of Enhancing Klaviyo e-Commerce Campaigns with A/B Testing Include: 

• Better omnichannel experience for customers
• Pinpointing messages that resonate the most with different customer segments
• Keeping brand messaging cohesive across all channels customers use Offering customers deals and products they are more likely to respond to.

In this piece, you will learn how to:

• Do A/B testing on your website
• Integrate Klaviyo with your A/B testing data
• Segment your email marketing campaigns based on your experiments
• Bonus: A/B test your email campaigns in Klaviyo

How to run your first A/B test

The key to running successful A/B tests is having an experimentation process. You can use this process to test and scale as your A/B testing program grows. Your experimentation process starts with understanding sample size needed for testing, data and business problems you need to solve.

Determine your sample size

The sample size needed for A/B tests can prevent many companies from running A/B tests.

Therefore, determining the sample size needed for your A/B test is the first step in your process. A/B testing tools like Kameleoon have sample size calculators in their dashboard. This way, you can check the sample size you need for your A/B test in the same place you run your A/B tests.

The traffic on the webpage you want to test is important. Even if your website receives over 10,000 visitors a month, the page you’re testing needs to have sufficient traffic for testing. If the page has low traffic, your test may need to run for a long time. If the page traffic is low, you can use on-site surveys or focus groups to get feedback from your customers.

Develop a hypothesis from business problems

With a page that has enough traffic for testing, you can dive into business problems you aim to solve. And create a hypothesis from that.

For example, if the page you want to test is a product page with high cart abandonment issues. The business issue you’re trying to solve here is high cart abandonment.

After you examine your analytics data, you can start narrowing down the reason behind your business problem. Apart from analytics data, look into behavioral analytics to confirm the issues your customers are running into.

Then you can create hypotheses based on what you’ve observed from the behavior of your customers. The hypothesis is the reason ‌your customers are behaving the way they do. This article on optimizing customer experience with user data gives you multiple examples of user insights that can help you form hypotheses.

Run your A/B test

After you’ve created your hypothesis, you can ‌run your A/B test.

From the example above, your hypothesis could be that removing the ‘Add to Cart’ button may improve the high cart abandonment rate. Or you could infer that customers are using the add to cart button ‌to save items for the future.

Log into your Kameleoon app. In the dashboard, click on ‘New Experiment’. Then, click ‘Create New Classic A/B’ experiment.

The Kameleoon App automatically creates two variations when you create an A/B experiment—the control and your variation.

Now you can ‌change elements in your variations to make it different from your control. For example, removing the add to cart button.

Finalize your experiment

To finalize your experiment, you need to choose the segment you want to target.

For example, you may only want to target customers who visit a particular product page. Or customers who take a specific action. You can also target customers based on language, location, and device type.

Select the traffic allocation for your experiment. Kameleoon automatically splits traffic on the testing page between your control and variation.

Finally, simulate your experiment to ensure all is as it should be. Then launch it.

Learn from your insights

Your variation could beat your control. Or vice versa.

Both scenarios are valid in a good A/B testing program. In the event your variation beats your control, ‌implement that new changes to provide better customer experience and increase your revenue.

If your control beats your variation, this could be because your hypothesis was wrong. This is a chance to go back to the drawing board and review your data (both quantitative and qualitative) to form a new hypothesis for another A/B test.

How A/B testing enhances email marketing campaigns

Just like in CX optimization, the level of sophistication of your email marketing campaign is determined by the extent of segmentation.

A lack of segmentation means your email campaigns are basically an email blast. As you get more customer data and build profiles, your campaigns begin to increase in sophistication and success. By the time you get to behavioral models, your email campaigns now respond to a customer’s buying behavior. And you have email flows around behaviors like abandoned carts.

A sophisticated email campaign uses predictive modeling for segmentation. You’re able to predict customer actions and put them into segments based on those actions. You’re using data from other systems in your company, such as A/B testing data or web analytics, to build out segments in your email marketing platform.

Email lists are commonly segmented into:

• Region
• Website behavior
• Device type
• Open rate

The problem with this method is that it is static and makes a lot of assumptions about your customers. These assumptions may be false and not even represent customers’ most recent behavior on your site. The only way to get confirmation is to send emails to your customers based on those assumptions. False assumptions will cost you subscribers.

On the other hand, website data from A/B testing and analytics software can help you build richer customer profiles for your email segments. Thus, enabling you to create a better omnichannel experience for customers.

To start using website A/B testing data in your Klaviyo email campaigns, you need a robust experimentation platform like Kameleoon.

Use experimentation data to inform customer communication

Kameleoon’s integration with Klaviyo gives you the ability to pull experimentation data.

You can then use this data to improve customer information. This data can help you improve and personalize customer messaging. As such, you can have a cohesive message across all platforms your company uses.

An example of a Kameleoon & Klaviyo campaign:

Imagine you have an online pet store. Your two best selling pet food products are chicken kibbles and beef bites.

Thanks to your data governance strategy, you are able to run an experiment to show dog owners both of these options as variants of a landing page. And you can find out which group of your dog owner customers are more responsive to beef or chicken.

With the Kameleoon + Klaviyo integration, you can use this information and segment your email campaigns. And can use this information to fuel communication. Like sending a discount code for chicken kibble to your customers in the chicken kibble dog owner segment.

Use Klaviyo’s predictive insight to create segments for website A/B tests

Klaviyo has a predictive insight engine that uses data you already have to elevate customer data and understanding.

Klaviyo can predict the amount a customer will spend in the next year, their next order date, how likely the customer is to churn and more.

You can start using this data to create segments for A/B testing on your website.

An example:

Klaviyo predicts when customers will make a purchase. You create a range of dates with this prediction. And bucket users who fall into this date range into a new segment for experimentation.

In your Kameleoon App, create a new segment where the condition targets email acquisition - traffic from your emails. Use page title to further narrow down the conditions of the segments. Since these customers are likely to make a purchase, you can use product page titles to further narrow down your conditions. And this segment is ready to be used for a test in upcoming experiments.

Adopt Klaviyo email and SMS campaign insights for website optimization

In Klaviyo, you can combine both email and SMS into one flow campaigns.

The results of these campaigns can give you insights into customer behavior. These insights can help inform both A/B testing and personalization efforts to solve business issues.

An example:

You run an SMS campaign where your loyal customers get a 20% off coupon. Your website analytics can tell you if this campaign has been successful or not. As users can use the link in the SMS to go directly to your website.

If after running this campaign, you notice the use of the coupon was low. This can prompt you into looking at session recordings to spot issues. And once you’ve spotted an issue, you can fix it for all your customers.

Other benefits of using A/B test data in Klaviyo campaigns

You can also use Kameleoon in the following ways:

How to Integrate Klaviyo with your A/B testing data

Kameleoon is an advanced client-side and server-side full stack A/B testing and personalization tool. Its unlimited, flicker-free A/B/n testing, AI personalization and real time reporting helps midsize to enterprise companies create world class omnichannel experiences for their customers.

It provides contextual, profile-based, omni-channel, rules-based and algorithmic targeting, plus content/product recommendation and testing capabilities for customers.

You can easily integrate Kameleoon with Klaviyo to create omnichannel experiences for your customers.

Integrating Kameleoon with Klaviyo

To push Kameleoon experiment data to Klaviyo, you need to have Klaviyo enabled on your website.

Then head into Kameleoon to activate Klaviyo. In your Kameleoon App, click on ‘Administrate’ in the left side menu. Then click on Integrations.

On the top right, click on the ‘New Custom Analytics’ button. A pop up will open:

Name this custom analytics, Klaviyo. And add your website URL. Hit ‘Next’. The new window will ask for a JavaScript code.

Kameleoon provides JavaScript code for Klaviyo. Copy the code and paste it into the new window.

Click on ‘Validate’ to finish integrating Kameleoon and Klaviyo.

Now, you have access to A/B testing data in Klaviyo.

How to create segments in Klaviyo

In Klaviyo, you can segment your email list by heading into your Segment/List tab in your account.

Click on ‘Create Segment/List’ in the top right hand corner. Then choose ‘Segment’ on the next page. Name your segment and add conditions.

These conditions can be location (country, zip code), custom properties added via an integration, actions taken on your website and more. Because you integrated Kameleoon with Kalviyo, you can now use custom properties like dog owners who buy only beef kibble, dog owners who buy only chicken kibble, and dog owners who buy both as conditions in your Klaviyo segment.

After adding conditions, click create and wait for your segment to populate.

How to A/B test your email Campaigns in Klaviyo

A/B testing your email campaigns is remarkably similar to A/B testing your website. The major differences lie in what you test, your sample size, and where you launch your A/B tests.

You can use the same process as a website A/B test with a few tweaks: Determine your sample size if you have a contact list of at least 1000 subscribers, you can start A/B testing your Klaviyo email campaigns.

The key is to use the 80/20 rule. Use 20 percent of your total number of subscribers for A/B tests. 10 percent for your control and the other 10 percent for your variation.

Then ship the winning campaigns to the remaining 80 percent of your subscribers. This principle allows you to shield a majority of your subscribers from negative treatments. While still having enough subscribers to get statistically significant results.

Use a sample size calculator to find out how many subscribers you need per variation.

What can you A/B test?

There are many things you can test in your email A/B tests. You can test‌:

• CTA copy and color
• Subject line
• "From" name
• Offer amount
• Visual content (images, gifs and videos)
• Layouts
• Brand colors and voice

The list is endless. Usually, you use open and click-through rates to measure the success of your email A/B testing.

Run your test

Once you’ve developed a hypothesis based on a business problem, you can set up your test.

On your Klaviyo page, head to ‘Campaigns’. Then click on ‘Create Campaign’. Then ‘Email’.

Name your campaign and select the list you want to send your test to. Then click on ‘Continue to Content’. Then add your subject line, preview text, sender name and address.

Create the first version of your A/B test by adding text, images, and links to your email. Hit ‘Save Content’. Go to the bottom of the page and click ‘Create A/B test’.

This automatically creates another variation of the first email you created. You can now edit the second variation to compare it to the first.

Keep the difference between your control and variation to one element. This way, you can be sure of which element is responsible for the lift you got. It can be harder to pinpoint this if you test two elements at once.