How to do Wix A/B testing for your online store

Wix is an easy-to-use website builder that helps businesses launch and run websites with no technical requirements or developers. The platform also includes basic A/B testing functionality with its “test site” feature.

But while Wix’s built-in A/B testing allows businesses and online stores get started with experimentation, it’s simplicity and lack of advanced measurement, targeting, and segmentation features prevents teams from scaling their testing program and implementing conversion rate optimization techniques.

This makes it impossible to:

• Scale your testing program to match your growing business
• Keep improving customer experience
• Increase revenue

In this article, we will show you how to get started with Wix A/B testing. Then, you’ll see how you can scale your experimentation program with more advanced tools like Kameleoon.

How to do Wix A/B testing for your store

Since Wix offers a native A/B testing tool for stores built on their platform, running an A/B test is easy. Steve Pogson, Founder of First Pier, an eCommerce marketing agency, agrees:

To start Wix A/B testing, you need to create a process that allows you to test variants and implement the findings.

1 Determine your sample size

The first step in running an A/B test is to determine your sample size. The traffic on the page you want to test affects how long your A/B test will run.

Testing on a low traffic page will take longer than testing on a high traffic page. And running your test for too long can contaminate your results, as most test scripts don’t last longer than 30 days. To calculate your sample, use this sample size calculator.

Note that dedicated experimentation tools like Kameleoon also come with calculators specifically for this purpose.

2 Find website problems and create hypotheses

Before you start A/B testing in your Wix store, you need to identify problems with your site. Fixing issues with your website via A/B tests can lead to quick results that increase revenue.

Use data from your analytics software to pinpoint problems that need further investigation. Complement that with user testing data to further narrow down what the problem is.

Here is a basic list of elements you can test on your page:

• Test on-page elements
• Website headlines
• Call-to-actions
• Hero images
• Pop-ups
• Pricing offers
• Lead capture forms
• Alignment
• Field labels
• Design
• Accessibility
• Device type (mobile vs. tablet vs. desktop)
• Font sizes

Example of creating a hypothesis

You may realize through both quantitative and qualitative data that many visitors who add products to cart on your website don’t buy the items, even with prompting emails. This may tell you that customers are using the Add to Cart button to save products they like. And it may be time to add a wishlist button for customers.

After identifying the problem, you can now make a hypothesis that adding a ‘wishlist button’ will prevent customers from using the ‘Add the cart’ to save items they like. And now you can test to find out if your hypothesis is right or not.

3 Create a Test Site

To create your Wix A/B test, you need to head to your Editor on your site’s dashboard. Click on ‘Site’ on the top menu. From the drop-down menu, click on "Create Test Site".

Now, you can edit your test site to change the elements on your page. For example, you can add the wishlist button to a product page. Then head over to the release manager to edit the test site’s exposure.

4 Manage your Test Site with Release Manager

After you create the test site, you will see a pop-up where you can view your test site or head over to the release manager.

You can also access the release manager from your dashboard, just under where you find the “Create Test Site” button.

In release manager, you will see your main and test sites. Creating the test site doesn’t mean your visitors will see it right away. You need to edit the exposure for your test site. Click on the Show More button (3 dots) on the test site. And then on "Edit Exposure".

Choose a number between 0 - 100 for your exposure percentage. Using the example above, you can set the exposure for your test site with your wishlist button to 50%. This means Wix splits your traffic between your live site as variation A and your test site with the wishlist button as variation B.

And your A/B test is live! Leave your test to run for an appropriate time, usually 2-3 weeks, to reach statistical significance and give you a clear winner.

5 Learn from your A/B test results

If your test site (variation B) beats your live site (variation A), you can publish it to all your visitors through the release manager. Go into release manager, click on the show more button. And click on the "Publish this version" button.

And, if the original site beats the test site, then it’s time to go back to the drawing board, as your hypothesis wasn’t right. You can also delete test sites in the release manager.

A failed A/B test is an opportunity to learn more through data and iterate better tests. [link]

6 Understand the limitations of Wix A/B testing

The native A/B testing tool in Wix is a great way to get started with testing. It is easy to use. However, its features and functions are highly limited.

Basic A/B testing only

Because Wix only allows one test site per website, you can only run a test with just one variation.

This type of test is restrictive, as you cannot run experiments with two variations against a control page. When you create a hypothesis, you may create two slightly different variations to test against your control.

Say you want to add a Wishlist button to product pages in your store.

Ideally, you want to add the button and test its placement. With Wix’s testing platform, you can only create one variation at a time. You’d have to test adding the wishlist button as a variation. Then create another test later to experiment with the wish list button placement.

This increases the time your store spends on running experiments. And how long it takes your site to implement changes to provide a better experience for customers.

Limited testing capabilities

In its native testing platform, Wix can only handle testing on one page.

Testing one page at a time restricts what you can test. And increases the time it takes to find insights that solve user experience issues on your site. This delay may damage customers’ perception of your brand.

Remember the example of adding a Wishlist button to product pages? Using Wix’s testing platform, you can only test one product page at a time.

Testing all your product pages isn’t an issue if you only have a few product pages. But it becomes an enormous problem when you have hundreds of product pages.

Alternatively, you can test one page and implement the changes for all product pages. But this approach leaves you right where you started if the problem you’re trying to solve persists.

Lack of proper targeting

One major drawback in using Wix’s testing platform is its inability to target different segments of your audience.

Wix’s testing platform is incapable of targeting visitors in an A/B test based on location, device type, and languag—never mind by transaction history or on-page activities. This makes serious A/B testing impossible..

For example, if your store wants to try increasing loyalty by pushing a discount on the homepage to existing customers.

You would not have the capability to target website users who are logged in or previously purchased on your site. This severely limits the store’s abilities to retain customers, and could cost you revenue in the long term.

7 Scale your Wix A/B testing with Kameleoon

Kameleoon is an advanced client and full-stack A/B testing and personalization tool with a native integration with Shopify Plus. Its unlimited, flicker-free A/B/n testing, AI personalization and real-time data reporting help mid-size and enterprise brands create world-class online shopping experiences for their customers.

It provides contextual, profile-based, omni-channel, rules-based and algorithmic targeting, plus content/product recommendation and testing capabilities for customers.

Wix website owners can easily connect Kameleoon to their stores and start optimizing their customer experience. Installing the Kameleoon script on your Wix site To start testing with Kameleoon, you need to first install the Kameleoon script in the head tag of your website.

Log into your Kameleoon App, click on ‘Sites’ under the ‘Administrate’ button on the left-hand menu. It will show the URL of the website you entered when you signed up. Click on the website card.

Click “Setup”. Follow the prompt on the new page to complete your installation.

Running your first test with Kameleoon

Running tests with Kameleoon is fast and convenient.

Say you want to test adding a wishlist button to prevent customers using the Add to Cart button to save products they like. And you also want to get the optimum placement of the Wishlist button so customers will see and use it.

Here is how you can run this experiment.

1. Log into your Kameleoon App.
2. Click on New Experiments
3. Select the classic A/B test option to launch the graphic editor.

You can start editing your variation as Kameleoon automatically creates two versions: your original and a variation. To create a second variation, click on the ‘+’ icon. You now have two variations and your original. Edit your new Wishlist button in Variation 1. Add the button to Variation 2 and change its placement.

Allocate traffic to your variations.

Choose segments and goals for your experiment. Simulate your experiment to confirm everything is as it should be. Launch your experiment. Triggers and Segments in Kameleoon Sometimes, your experiment will not target every customer who visits your website.

When setting up your experiments, you can choose which segments of visitors to target. In the above example, your segments for that A/B test will only be people visiting your product pages.

With the store trying to target previous customers, their experiment segment will only target customers based on on-page activity (completing a purchase).

In the targeting section of your experiment, you can choose from an existing segment. Or create a new segment for your experiments.

What can you test with Kameleoon?

Everything that can move the needle in providing better experience for your customers and increasing revenue for your business. Kameleoon offers everything you need in an optimization solution:

• A/B/n testing
• Multivariate testing
• Personalizations

There is no limit to experimentation with Kameleoon. With Kameleoon, you can run different A/B tests to improve customer experiences and outcomes.

Scale your experimentation by testing multiple elements at once with Kameleoon’s multivariate testing. And use Kameleoon’s advanced personalization at scale engine to give your customers a more personalized touch.

To find out if your store can increase conversion rates and provide better customer experience with an advanced experimentation tool like Kameleoon, book a demo call.