Split Testing Shopify Product Pages: Start the Journey to Optimize Your Best-Sellers

Guessing is not a sound business practice. If you’ve been around the eCommerce block for a while, you’ve seen the evidence.

So, when it comes to your Shopify product pages, the moneymakers of your online store, your growth strategy has to take a scientific approach.

Even when you use best practices or the “secret weapon” of your competitor, the results could still be poor on your Shopify product page, leaving you wondering, “What am I doing differently?”

Split testing your product pages is how you can discover the exact changes that work for your unique store, offering, and audience. This is how you optimize them to entice buyers to open their wallets in order to maximize your revenue.

In this article, I’ll show you why your product page is a fantastic candidate for split testing and 3 ways to do it without hurting your business.

Let’s begin.

Why Focus on Your Shopify Product Page?

Sure what we do has to make commercial sense, but it’s never the starting point. We start with the product and the user experience.

Steve Jobs

As a Shopify store owner, the metric you’re most concerned about is conversion rate. If it isn’t, it should be.

If you take a look at the pages in your Shopify store, you’ll find that

• Conversion on the homepage is clicking on a product, collection, signing up for email updates, etc.
• Conversion on the collection/category page is clicking to a product page
• Conversion in the navbar or search bar is locating a product (and yes, the product page)

Even if you have a campaign running and sent out an email offer, conversion happens when recipients accept that coupon and use them after deciding through, you guessed it, a product page.

The entire ecommerce sales funnel is pointed toward the product page. Why? Because that’s where the buying decision happens, that’s where revenue generation begins in earnest.

So, every element on it—copy, buttons, product image, navigation—has to work together to get a visitor to click the cart button. To do this, the product page must:

• Inspire trust in your brand
• Provide sufficient information about the product and company
• Seize attention and guide it towards the CTA button
• Persuade browsers to pay for the product, and even
• Motivate them to check out related products.

How do you achieve all these with visual elements and product descriptions on such a high-stake page as the product page? That’s where testing helps out.

As the de facto landing page in several marketing scenarios, you may be nervous about making changes to your Shopify product page.

You’re not alone. Most marketers and merchants are nervous too, that’s why many go with what’s considered the textbook look and feel of the product page and instead focus their attention on other parts of the sales funnel, hoping those help the conversion rate.

Imagine your product page is built for someone who followed the path you think most buyers follow:

homepage > category page > product page (maybe a short detour to the “about” page).

When someone who didn’t follow that route meets your product page, it won’t address the thoughts and objections they have in their mind.

And believe it or not, visitors who don’t follow that route are many (even a majority). For instance,

• A very specific Google search that brought them to your product page, or
• A forgotten browser tab from last month, and now they’ve forgotten why they were there in the first place.

You need to focus your optimization energy on the product page; give yourself the opportunity to test a variation of your page that speaks directly to your specific target audience.

In the words of Rishi Rawat, the Shopify product page guy, “this is where our hero chooses ‘buy’ or ‘bye bye’”.

What is Split URL Testing? When Should You Split Test Your Shopify Product Pages?

Split URL testing is a type of test where you pit two substantially different versions of the same way page against each other, sending half of your traffic to either of them to see which performs better at converting.

The keyword here is substantial changes. That’s one way it is different from A/B testing. The other way is that these two pages usually have different URLs, for example,

“https://yourwebsite.com/products/dog-chew-toy” and “https://yourwebsite.com/products/dog-chew-toy?view=plus”

This way you can test different layouts of your product pages. You can also split test Shopify Plus themes and Shopify product page templates.

You should think about split testing when:

You Have Unoptimized, Poor Performing Pages

If your product pages aren’t performing as you would want them to, or you’ve observed an unusually low conversion rate in the product-page-to-cart-page section of your sales funnel, start with split testing to isolate issues until you find the culprit.

There is likely an experience that is actively stopping or derailing the purchase journey of buyers, and split tests can help you find it. But remember to start with checking your analytics data and doing customer research.

In Joshua Uebergang’s book Shopify Conversion Rate Optimization, he suggests these questions for your customer survey:

• What almost stopped you from buying from us? The best question a merchant can ask on their thank you page.
• What other options were you considering before buying from us? Another good question post-purchase to understand the psychology of options and competition.
• It seems you didn’t complete your purchase, can you share why not? Great for when people leave the cart page or stay on it for more than 20 seconds.
• What other information would you find helpful on this page? Trigger after 20 seconds to show only to an engaged visitor. Good for product pages to fill in information gaps.

You Are Working with Two Different Shopify Themes

With two themes, you might be wondering, “Which converts better?”

Split testing your Shopify themes will help you put that to rest.

An important point to note in split testing themes:

Swap themes every night at midnight to minimize the possibility of a user seeing a different theme during their browsing session.

JoeIdeas, Shopify Expert in the Shopify Community Forum

You’re Testing Prices

This works well with split testing since it will be easy to directly track conversions in an analytics tool like Google Analytics for the two different URLs.

Revenue tracking is one of the more challenging and error-prone endeavors in experimentation. With split testing pricing—especially at mass—you can avoid the challenges of revenue tacking.

But there’s a common concern with price testing and that’s whether it is ethical. That’s a different story. We explored that in detail here.

You’re Testing Different Cart or Checkout Behavior to Reduce Abandonment

A/B testing a checkout process may not be feasible due to its complexity.

Anything that overly complicates the setup and tracking of an A/B test would probably benefit from a split URL configuration.

Split tests also tend to conclude faster than A/B tests because of the nature of the changes being tested (significant). This is why split testing is a recommended starting point for un-optimized product pages where baseline conversions are much lower than industry standards. You can spot some big wins, early on.

A Quick Note: Split Testing Isn’t A/B Testing

A quick search on Google will show you a lot of marketers use “A/B testing” and “split testing” interchangeably. But they are not the same thing — at least on a fundamental level.

Here’s the difference:

A/B testing is when you change one or more elements on a page and run this change against an original version of the page to see how it affects the conversion rate.

The page with the changes is usually called the “challenger” while the one without is the “control”.

Both the original and the control are hosted on the same URL.

With split testing (AKA split URL testing or redirect experience) you send your traffic to two completely different versions of a page (which are hosted on distinct URLs).

In both cases, the goal is the same: to see which version performs better.

Split testing is best suited for radical changes in page design. You can use it to test different layouts of your Shopify product page to learn which performs better in converting web visitors.

When Should You A/B Test Your Shopify Product Pages?

Once you’ve eliminated what obviously doesn’t work with split testing, A/B testing helps you zero in on the finer nuances of choice.

While split testing reveals the design or layout that works best for your Shopify product page, A/B testing shows you the elements to add, change, or remove to fine-tune your conversion rates.

Because A/B testing is just that — this vs that, testing small, controlled changes.

That way, you learn what your traffic prefers on the product page as opposed to what split testing your Shopify product page does for you: shows you what is actively stopping your web visitors from buying.

A critical point to note is that you run the risk of reversing some of the gains you made with split testing if you add elements willy-nilly. Don’t guess your way to success.

It’s not enough to complete the big changes. You need to use that scientific approach on the finer details as well. Use Shopify A/B testing to get validation and assurance for smaller changes to your product pages.

LastObject, the Danish manufacturing company based in Copenhagen, worked with Ontrack Agency and learned from qualitative research that their users were concerned about the quality of the products and the shipping details.

The solution here was an FAQ section on the product pages. That makes sense, right? But how do you know for sure that won’t negatively affect the conversion rates? Are you bugging customers with too much information? Would it be better to just add a live chat feature so they can ask customer care directly?

This is why they tested it — to see if the FAQ section would increase conversion rates. This wasn’t a radical change to the product page, just an element added to help conversions.

The collapsible FAQ section and users could now have their questions and concerns addressed right on the product page. This boosted the conversion rate by 12.5% with an improved add-to-cart rate of 3.18%.

Another example is one Shopify product page A/B test Frictionless Commerce ran on Oliver Cabell’s website, the DTC premium shoe brand. They tested adding Oliver Cabell’s story on all product pages in a visible, non-distracting way.

Clicking on “WANT TO KNOW A SECRET?” opened a lightbox window with the story and even compared their price against competitors.

A version was also created for mobile devices since previous tests had revealed they respond quite differently to tests. This is an excellent example of the kinds of insights you can collect through solid ecommerce experiments.

After running for 3 weeks, the test showed a 12.46% lift in traffic to the checkout page. That was on desktop. On mobile, it was even higher at 21.77%. Together, they resulted in an 11.06% lift in revenue from that product page.

Want to do something similar? Let’s show you 3 ways you can split test your product pages.

3 Different Ways to Split Test Your Product Pages

Split testing your product pages on Shopify involves 3 basic steps:

1. First, you start with qualitative and quantitative research to find problems on page
2. Then you create a hypothesis, that is, a solution you believe can fix that problem, and
3. Finally, deploy the test.

There’s one final step that is a sound CRO tactic: learn from the test, document it, and use it to plan future tests. This is how you build an excellent data-driven ecommerce growth strategy.

In the 3 methods of split testing I’ll show you below, all the steps above are necessary. What differs is the execution (or deployment) step in number 3.

Let’s check them out:

1. Client-Side Split Testing

Client-side split testing is the type that’s executed on the visitor’s browser. It is the easiest and most beginner-friendly option, as you get to use a WYSIWYG editor in your split testing tool to make the changes you want.

From there, it’s as easy as setting the parameters, launching, and waiting for the results. For this, you want to use a Shopify testing tool that integrates smoothly with your Shopify store.

2. Redirect Test

This is the method that puts the “split URL” in the name of this test. Here, the variants of your test exist on a different URL. Like I mentioned earlier, this can be something like

“https://yourwebsite.com/products/dog-chew-toy” and “https://yourwebsite.com/products/dog-chew-toy?view=plus”

Notice the “?view=plus”?

That’s Shopify’s alternative template URL parameter. It lets you show your visitor a different product page template on your product pages. You can show one half of your traffic one version of your product pages and the other half a different version. All set up on a quality split testing tool.

In regular A/B testing, the original and challenger exist on the same URL. Your A/B testing tool then chooses to show either one to the visitor. If it’s the challenger, then it quickly changes the content of the page to that of the challenger.

A solid tool does that without flicker. In redirect split URL testing, the entire content of the page is already what you want the visitor to see, so no need for hiding or adapting the content to the specific instance.

3. Server-Side Split Testing

Server-side testing comes in handy when you want advanced testing functions, like when you want to test apps.

For instance, you have an app that adds extra functionality to your product page such as a stock counter that shows the limited availability of your products. Let’s say you have two of those, you’d want to know which performs better.

Server-side testing—a type of testing that’s performed right in the liquid.code file on Shopify—is the ideal fit for this. But you may need a developer to execute this.

You can get started on one of these methods with one of the best A/B testing tools for Shopify: Convert Experiences. It seamlessly integrates with your Shopify store, giving you the freedom to run A/B, A/B/N, Split, and MVT tests with a custom Shopify A/B testing app. (It actually has over 100 integrations with popular ecommerce tools).

7 Mistakes to Avoid When Split Testing Shopify Product Pages

While you’re excited to dive into making smarter decisions about the growth of your Shopify store, make sure you don’t make these mistakes that plague new testers:

1. Not Going for the Low Hanging Fruit First

Go for the big impacts — those changes that directly affect your bottom line. It’s easy to test the CTA button color or see if a sticky CTA would give you a lift. But that’s not what split testing is about.

You want to test those big changes you’ve been scared to implement because you weren’t sure of how they’ll affect shoppers’ responses.

Changes like a new Shopify product page template are big impact, low hanging fruits you should test and improve first.

2. No Pre-Test Analysis

Do you have enough traffic for the test? What’s the sample size you need to hit statistical significance? And how long do you have to keep it running?

In CRO, it’s proper to do the math before you hit the tool, so you don’t time and money. You can use an A/B testing significance calculator for this.

Many tests fail because they don’t have the required traffic or sensitivity to produce usable results. Pre-test analysis before split testing a product page on Shopify will let you know if the effort is worthwhile and how long the test should run.

3. Using the Wrong URL

In split testing, where the URLs are usually different, it’s easy to make a mistake with this. Make sure you have inputted the correct URL and you’re not pointing to a staging page.

Both pages have to be live so that when your visitors visit them, they actually experience the test you built. It sounds simple to figure out but it happens. That’s why QAing is a vital step.

4. Making Changes While the Test is Running

If something isn’t broken in your split test setup, then you don’t need to touch anything while it’s running. Don’t edit the page mid-test — whether it’s the copy, design, or position of elements. If you do that, you’ll skew the results. Let it be; let it run and give you unadulterated results.

5. Stopping the Test at the Wrong Time

When should you stop a test? A lot of people rely on their A/B testing tool to tell them. But you can’t always depend on the information you’re getting.

Sometimes your tool tells you your result is statistically significant after running for a short time and it may make sense then to stop since you have a “clear winner”. But that may be a mistake.

You should only stop a test in that instance if it’s met the required sample size based on your pre-test analysis. Anything less and you can’t trust your result isn’t due to some abnormality. For instance, a spike in sales during a holiday or the payday effect.

As a rule of thumb, let your split URL tests run for a month, hit your target sample size, and wait for a 95% confidence level.

6. Peeking

Peeking is checking out the test to see how it’s performing. This is okay in the first 24 hours, to make it’s running and everything is set perfectly. But after that, you don’t want to check the tests to see which is leading, or worse, make a decision based on what you see.

You want to wait for the 3 criteria I mentioned in mistake #5. Don’t read results before that’s done. Even experts are guilty of this, so fight the temptation to find out which version of your product page is winning.

7. Not Documenting the Test

Making data-driven decisions should be part of your ecommerce business. To make that happen, you need to learn from past tests and build on those learnings to optimize further.

Also documenting helps you avoid yourself or another member of your team repeating a test you’ve already done.

In your test documentation (which you’ll add to your learning repository), include:

• Information about the product page
• The hypothesis you’re testing
• The outcome of the test
• What worked and what didn’t, and more

Split Testing on Shopify vs Shopify Plus

While Shopify is the basic version of the ecommerce platform, Shopify Plus gives you greater flexibility and functionality to accommodate the increased demands of bigger, enterprise-level online stores. Think of it as the premium version of Shopify.

Almost everything is customizable and the admin panel lets you take on more detailed reports, shipping, sales, and inventory management.

This extra flexibility and customizability are what make Shopify Plus better for split testing. You see, there’s a challenge with testing on basic Shopify because you don’t have as much access to the code as you would with the Plus version.

For example, split testing themes on Convert is possible only for Shopify Plus accounts where the preview banner can be hidden. Do you know that banner that shows that you’re on a preview template?

You can take it off with a simple CSS code. This superior access to your code makes split testing easier and more accurate.

Also, if you want to run a test on Shopify checkout, Shopify Plus merchants are set up to get much more accurate results. Why? Because testing tools use a first-party cookie that’s linked to the store’s domain name.

What this means is that when the visitor visits a different domain, the cookie stops working. For those without a Plus account, the checkout links the visitor to checkout.shopify.com. Your testing tool stops tracking that visitor because technically, they’ve left your domain.

So, if you’re missing features or want to get serious about conversion rate optimization, Shopify Plus will better support your goals.

What Can You Expect by Split Testing Shopify Product Pages? Examples From the Wild

Let’s take you through 3 examples of test results ecommerce store merchants got from split testing their product pages. You can also get inspiration about what to test and how to think of your store today to find optimization opportunities later.

1. Dr. Squatch

Personal care DTC brand, Dr. Squatch, hired SplitBase to optimize their store’s experience.

In one of the tests they carried out, they learned from analytics data that customers bought more than 1 bar of soap at a time. And this matched what they discovered in customer research.

So, they tested adding a quantity field on the product pages, thereby helping people to decide if they wanted more than 1 of that product at a time.

This turned out to impact their average order value in a great way. This test showed an increase in revenue per user by 54%.

2. Changing the Mobile Version UX of the Product Page

An ecommerce retailer boosted its revenue by 17% by changing the mobile user experience on its product page.

They pinpointed these friction-causing UX problems with user research and came up with a hypothesis that fixing those problems specifically for mobile devices will boost conversion rates.

Sort of a no-brainer, right? But think about how many merchants aren’t actively researching friction in their customer experience journey on a device-specific level, especially on the product page.

This brand found that 80% of its users came from mobile devices and 95% of them found the most popular product, whose product page was essentially the landing page. And that the product page performed poorly in getting this traffic to the cart page.

Then they created a variation of that page with those issues fixed (and a few queued up for A/B testing). They split tested and besides the 17% lift in revenue, this was the result:

• 14.43% conversion rate lift
• 9.9% add-to-cart rate increase

3. Silver Fern Brand

Silverfernbrand.com’s CRO agency, Frictionless Commerce, helped them get a 32.38% lift in sales of their bestseller Ultimate Probiotic Supplement. And what they did was optimize the product page in a genius way. Here’s how that went…

They hypothesized that if they get users to tell them what health issues they were experiencing and show them positive customer reviews directly related to that issue, that will get them excited to buy the probiotic and increase the number of conversions.

So, they tested adding a brief quiz at the top of the product page.

After picking the issues and clicking on “What’s the solution?”, they get reviews related to that selection.

In two weeks, this showed that the quiz can boost revenue by 32.32%.

Key Takeaway

Split testing your Shopify Plus product page tells you what you don’t know about your customers so you can figure out the most profitable way to present your products to an online audience.

This is the antidote to guessing, cart abandonment issues, and the recipe for improved add-to-cart rates.

You can use it to find the best among multiple Shopify product page templates and themes, and also the unique page layout and functionality that could mean the difference between a lackluster bottom line and setting new growth records.

We’ve shown you 3 ways to split test your product pages and this can all be done with Convert Experiences. Get a 15-day free trial to run split tests on your Shopify Plus store with a testing tool that smoothly integrates with it.