Hi, In order to better understand what you'd like to get out of our website, we track the URLs you visit in a session. This information is only used to customize future communications with you. Do you consent to allow this tracking?
I Agree I Disagree. Disable Tracking
5 Ways Digital Marketers Use Call Tracking to Grow Their Business
Angela White
Ed-tech enthusiast with a passion for writing for the consumer market in...
Read More About Angela

The Ultimate A/B Testing Quick Guide for Shopify eCommerce Stores

June 1, 2020 –

Every eCommerce store on the Shopify platform is unique. It doesn’t matter if you are using a quiz builder to sell lead quizzes or you sell retail goods. What matters is that not all eCommerce stores share the same group of customers.

Finding out what works best for others and implementing it in your shop often bears little to no results at all. Why? Your shop attracts a different group of people, meaning that they have specific needs and expectations.

But does it still leave you wondering how to leverage all your store’s traffic and turn it into sales and revenue? The answer lies in a marketing experiment called A/B testing or split testing. With the help of A/B testing tools, you will be able to fine-tune your eCommerce pages for high conversion rates.

A/B Testing – A Quick Summary

A/B testing is an easy and convenient way to find out which page layout captures the most sales. The principle is simple. You build two different pages. For instance, you can place a Checkout button in two different places or use different CTAs. Then, you serve those two pages to your potential customers.

One part of your traffic will be served with “page version A,” and the other will be served with “page version B.” Once you are done with testing, you will compare the performance of those two pages.

In our use case scenario, page A is a page on your eCommerce website, and page B is a new page that you hope will do better than page A in terms of conversions. You can repeat this process until you are satisfied with page B’s performance. Now, you can take down page A, replace it with page B, and you are done.

When Not To Run A/B Test

Let us save you some time in advance. An A/B test is not one of those one-size-fits-all marketing solutions. Your online shop has to have a substantial amount of traffic for A/B tests to be reliable. How do you know if your shop qualifies for A/B testing? This sample size calculator can help you a lot.

Input your current conversion rate In the “Baseline Conversion Rate” field, as well as how much you want to increase it in the “Minimum Detectable Effect” field.

Let’s say your conversion rate is 4%, and you want to increase it by 20%.

The tool says you need a 9,683 sample size per variation. So, you need 9,683 visitors for Page A and 9,683 visitors for page B. In total, you need 19,366 visitors. That’s quite a lot, especially for people that have just launched their online shop on Shopify.

This doesn’t mean that you need to abandon split testing altogether. Monitor the traffic on your pages, and, as soon as it crosses the threshold, start running A/B tests.

Standard users should not engage in A/B testing because of the problems related to cookies. Standard users use the checkout.shopify.com domain as a default checkout funnel, and they can’t change it. Since there is a lot of cross-domain traffic, A/B testing tools rely on data in the cookies.

The problem is that some users block cookies, which makes A/B test results incomplete and should not be used for making important business decisions.

Easy A/B Testing

With Shopify, there is an easy and hard way to do A/B testing. For now, let’s stick with the A/B testing for everyone. To do it in a laid-back fashion, use one of the page builders with built-in A/B testing capabilities. This should not come at a huge expense because page builders come with monthly subscription plans. One month is more than enough for you to complete the A/B test.

There are plenty of page builders and other solutions on the Shopify app market that can help you streamline A/B testing. Shogun page builder, Neat A/B testing, Always Be Testing – A/B Splits, PageFly page builder; these are just a few of the paid solutions you can use with minimal technical knowledge to run basic split tests.

We say easy A/B testing because it really is. With one of these tools, you can select a page, choose a pre-set objective, customize the settings, and run a test. You can do it in a matter of minutes and come back in a few weeks to see the results.

Advanced A/B Test Shopify’s Checkout Page

You don’t have to be an IT guru to run A/B tests on your Shopify online store. Convert Experiences can help you streamline your A/B testing efforts. More importantly, you can easily integrate it with Shopify and run A/B tests on your checkout page and more.

Convert Experiences Tracking Code
Convert Experiences Tracking Code

Integration is straightforward! After you create a new project in Convert Experiences, copy the Tracking Code and head to your Shopify dashboard. Click on Sales Channels > Online Store > Themes > More Actions > Edit Code, select the theme.liquid file and paste the code anywhere in the <head> section of the page.

Additionally, you need to add the code for every page you want to run through the A/B test. It’s easy. In your Shopify dashboard, click on Settings > Checkout (go down to) > Additional Scripts, and paste the following script:

<script type="text/javascript">
  if (typeof _conv_host == "undefined") {
  window["_conv_prevent_bodyhide"] = true;
  (function() {
    var _conv_track = document.createElement("script");
    _conv_track.src = "//cdn-3.convertexperiments.com/js/XXXXX-XXXXX.js";

    document.getElementsByTagName("head")[0].appendChild(_conv_track);
  })();
}
</script>

Make sure to replace the XXXXX-XXXXX with your Convert Experiences Project ID.

Shopify Plus users don’t need to do this as they can directly edit the checkout.liquid file. If you have a Plus account, then you can copy the Convert Tracking Code and paste it in the checkout.liquid file inside the <head> section (Sales Channels > Online Store > Themes > More Actions > Edit Code).

Go back to your Convert Experiences dashboard, open Project Settings, and add the URL of the pages you pasted the tracking code in. Click on the “Verify” button. If the light is green, it’s a good sign; you’ve integrated Convert Experiences with your Shopify site.

If your checkout page is on a different domain, things will get slightly complicated. You will need to spend a few more minutes to configure your setup and forward tracking cookies to the checkout domain. Once you are done, you are ready to do some A/B testing.

Making A/B experiment pages is easy. Convert Experiences will host your experiment page. All you have to do is enter the URL of the page you want to optimize, name your experiment, and make the variation you see fit (the Convert Experiences page editor will enable you to edit the page without coding at all).

Convert Experiences Page Variation
Convert Experiences Variation Editor

Once you name your variation and save it, you can set the audience and goals of the experiment. Come back in a few weeks or a month, and you will see an in-depth report page.

If you run an ecommerce business on Shopify and want to set up more complex A/B tests, give Convert Experiences a go! It’s free to try for 15 days.

Quiz Builder as an A/B Test Alternative

If your traffic is still too low to benefit from the A/B test or you don’t want to invest in paid A/B tools and page builders, you can always directly engage your site visitors. There is an easy way to solicit feedback from your customer base without breaking a sweat.

Did you know that you can embed almost anything on your Shopify eCommerce website? You can create an online quiz with a quiz builder tool, embed it on the Shopify website, and ask your visitors to provide you with feedback.

Nothing boring and intrusive; a couple of questions should be enough to discover what they like and don’t like about their shopping experience at your online store.

With modern quiz builder software, you will be able to customize the quiz so that it fits perfectly into your online shop design and experience.

In A Nutshell

Running A/B tests on the eCommerce website is paramount if you want to increase conversions rates and revenue. As you can see, there are several ways to do it. While online quizzes can help you gauge your visitors’ opinions for free, you have other options.

In the end, it’s your choice.

You can either use apps and page builders that have A/B test features or give a specialized and ultra-fast A/B testing tool such as Convert Experiences a try (it’s free for 15 days).

Originally published June 01, 2020 - Updated June 19, 2020

15-Day, Full-Access Free Trial of Convert Experiences.

Watch Icon
Quick form submission.
Credit Card Icon
No credit card needed.
Headphones Icon
Support available.
Your email ID is used to send you the registration and login details for the app, (up to 7) suggestions to use the application, and 1:1 emails from buyer journey experts who help you choose the right plans.
You also give Convert consent to send you top CRO content in a monthly newsletter*
You can always change your preferences later.

We've Improved Conversions for
5000+ Leading Brands

Check Icon
You're Almost Done.
Managing Marketing Team
Managing Tech Team
Hypothesizing Experiments
Coding and QA of Experiments
Convert is committed to protecting your privacy and sees GDPR, CCPA and other Privacy Laws as an opportunity to strengthen our commitment even further. We don’t collect & process users’ personal information beyond what is required for the functioning of our services, and this will never change.

You're successfully signed up for the 15-day
Convert Experiences free trial.

You'll be re-directed to the app in a few seconds.
You will also receive a welcome message to your registered email with your login access.