Blog/A/B Testing

Did you know there are over 50 A/B testing tools on the market?

Yes, that many.

With so many options available, how do you know which one will fit your needs in 2023?

You don’t need to break the bank or go with the category leader if you want your testing needs met. Consider what kind of optimization challenges you face on the regular – the best A/B testing software for you should match your needs and budget.

How do you narrow down ‘the one’ from all the tools on the market?

In this blog post, we’ll outline the factors you should consider when making your decision and share our top picks for the best A/B testing tools of 2023.

Let’s help you out!

But first…

Quick Recap: Understanding the Optimization Puzzle

This State of Experimentation Maturity report identified 4 pillars of experimentation maturity. These are core aspects of experimentation that helped organizations across various industries succeed at experimentation while scaling their programs.

It’s interesting what we’ll learn here. Those pillars are

1. Process and accountability: This includes the process for ideation and prioritization, experiment design, how success is measured, etc.
2. Culture: What’s the organization’s general attitude towards experimentation? How much C-suite support is present?
3. Expertise: Time, resources, and skill sets. And finally…
4. Tools and Technology: The tech stack powering your experimentation.

Although this research was conducted a few years ago, the fundamentals remain unchanged. Tools are but a very small piece of your optimization puzzle.

If you’re reading this to find the best A/B testing tool that can singlehandedly take your testing program from 0 to 100 or okay to fantastic, then consider this a reality check.

What then should be your major focus?

Arguably, people and processes take the cake when it comes to experimentation success. The brands and agencies at the top of their CRO game take training very seriously. And so should you too:

You should invest in your team more than just as a best practice in A/B testing; it’s often an expert-prescribed priority for organizations who want to take their optimization game to the next level.

This training goes beyond basic A/B testing tutorials and also includes research, data collection and analysis, critical thinking, experiment design, statistics, QA, stakeholder communication, and so much more.

Let’s put a pin on this idea because we will revive it later.

One stream of this education mix should come from your A/B testing vendor. That’s something to keep in mind in your search for the right tool for your team.

The right tool will provide education that corresponds to understanding what practices and frameworks offer the most efficiency for high test velocity (which is a program objective that many of the best minds in experimentation, including Ronny Kohavi & Lukas Vermeer recommend) in the context of that particular testing platform.

Since people and processes make or break your program with a greater degree of influence than your choice of A/B testing software, an idea that is gaining popularity in the industry is to spend as little as 20% of the optimization budget on tools and the rest on education, recruitment and Ops.

For example, agency WeTeachCRO believes that tool spend can and should be far less than 20%:

Conventional wisdom says not to spend more than 20% of your total CRO budget on tools, keeping 80% spare for people – people after all are the ones that will make the real difference to your programme, whereas the tool is merely an enabler. But putting that “wisdom” aside, our experience is that most clients can (and should) spend far less than 20%.

For the majority of our clients, their tool costs are <£500 / month with a people cost 10-15x that, putting them more in the 5-10% bracket. Starting out with a free tool like Google Optimize should get you off the ground, with a view to graduating up to something that is a few hundred pounds per month once your test complexities exceed its capabilities

Excerpt from 52 Questions Every Experienced CROer Had To Ask Once by WeTeachCRO

Now that we’ve gotten the right mindset in place, let’s move to the next crucial consideration in buying an A/B testing tool. Especially if this is your first A/B testing tool purchase.

If not, you can skip to outlining why you need a new tool.

Do You Have Enough Traffic to Test?

A/B testing doesn’t work for low-traffic websites. Before you begin testing, your website must have enough traffic to send to each variation of your A/B test. Without adequate traffic, your experiments will not reach statistical significance and may run for a long time.

In a typical A/B test, you need to drive thousands of visitors to each variation. At Convert, we recommend sending at least 5K visitors to each variation and around 1000 conversions to produce conclusive results of which variant is the clear winner.

Head over to Convert’s A/B testing calculator to find out how much traffic, conversions, and average order per revenue you need to run your A/B test. In the event you lack the required traffic to the A/B test, head over to our guide on testing with low traffic.

Once you’ve sorted out the required traffic to successfully run an A/B test, outline what’s prompting you to get a new A/B testing tool.

Start With the Problem

Figure out exactly why you need a new A/B testing tool. That goes without saying but it’s vital to have a structured approach to doing this. Here’s how:

Map Out Why You Purchased the Existing A/B Testing Tool

If you already have a testing tool in your stack, what led you to it?

Take note of this. It’s your point A and will help you get to point B, which is your next ideal tool. But we’ll make a few stops along the way.

Include objective items such as the features of the platform. Which ones attracted you and your team to it? Which ones retained you as a user? What did you like about this vendor in particular?

Don’t shy away from digging deeper. Understandably, there’s a strong emotional aspect in our decision-making process, but for the sake of a solid optimization program moving forward, you need to drill into your current tool.

Where Was Your Program When You Started?

Were you starting out in A/B testing and just wanted to get your feet wet? Did you hear about website optimization on a growth marketing podcast and had this tool recommended to you? Or were you on a free tool and wanted to amp up your testing beyond its limits?

What Guardrail Metrics Were You Hoping to Move With the Tool Acquisition?

Guardrail metrics are meant to keep you aligned with your goals and objectives. They alert you when things aren’t going quite as planned.

So, you definitely got the tool because you believed there was something you could change. What was it? Revenue? UX metrics like bounce rate and order rate?

Did You End Up Achieving Your Goals?

If not, then why? Did your testing program pivot? Or was the tool—GASP—a bad buy?

Answer these honestly and you’ll have a clear map of the problem. That’s your point A.

You didn’t find any problem at all? Or you don’t have the answers to some or all of these questions? Who are the actual buyers and the heavy tool users? Having an informal chat with them will get you the answers you need.

Because you don’t want to change tools just because you have the budget and someone wants a fancy new feature.

Features hardly ever solve problems. They can, at best, help in the execution of solutions. And the shift should make sense for you:

• Adoption-wise
• Vendor support-wise — If you have a great rapport with your existing vendor, this means something. It is having a partner on the journey.
• Roll out and ramp up time-wise — Days lost adjusting to a new tool are the days you won’t be testing, either at all or at capacity.

Bottom line: Decide to shift if and only if you know ‌the move will tackle all the elements that make up the problem you’ve identified, as well as plug holes and bring additional value to your people and processes.

The tech will catch up to whatever bells and whistles you think you need if those bells and whistles enable a real opportunity and not trends. Don’t rock the boat for one feature.

Ask the Right People

Once you have established that there is a real problem that can be solved by experimentation or by changing platforms and not something either frivolous that won’t be worth the effort or a chimera (the ghost of operational and procedural chaos manifesting as tool trouble), then move on to building your consideration set.

To build the consideration set, we have to arm ourselves with the right data to make a sound decision that covers all bases. And as with everything, the right data are your problems, not opinions.

You’d also want to avoid a common mistake most people make — going for the category leader. Category leaders are rarely the best at everything. Neither should you default to the same tools you evaluated last time.

Because experimentation is complex. You need a platform that will support your specific program objectives and your experimentation workflows. While fitting perfectly into your culture, since all the pillars of experimentation maturity have to work in tandem.

Be open to choices. Like Peep Laja says, they instantly improve the situation.

So, where do we begin?

What B2B Buyers Say About Buying Tools — Here’s Where They Start

We ran a poll on LinkedIn. We asked B2B software buyers, “What is the first thing you do when buying software that cost over $5,000 a year?”

They answered:

• 57% do a Google search
• 21% research known brands
• 14% start on G2 or TrustRadius, and
• 7% ask for referrals

Additional opinions included those who will do a combination of all four. Although not a significant sample, it’s fairly representative and gives us a glimpse into buyers’ preferred method to begin their tool search.

So, the tool with the strongest SEO wins? Not exactly. We’re taking this data with a massive pinch of salt. It illustrates that when we don’t have a clue about something, we’re kind of wired to turn to Google first.

While that’s a springboard for your search—giving you a list of A/B testing tools to consider—you need a more informed strategy. One that comes from the bottom of this poll: Ask the right people.

What We Recommend

Consult people who have used A/B testing tools extensively, your team, and also see what competitors or other players in your niche use.

Ask People in Your Network Who Test Regularly

Start with

• LinkedIn: Look into groups for conversion rate optimization pros. Some are more specific for SaaS, marketing professionals, and Shopify. Or make a post with the right hashtags (ex: #CRO, #abtesting #experimentation, and #optimization), tag the frequent A/B testers you know, and publicly ask for recommendations.
• Slack communities: Request recommendations on Slack communities for marketers such as CRO Growth Hacks, Marketers Chat, Growmance, and Measure.

You’ll want to avoid

• Agency contacts: At least at this stage because most‌ have tie-ups with tools or their favorites that may or may not involve financial kickbacks and compensation. This is neither good nor bad. It’s just how the space is right now.
• Google results: Avoid Google search results for specific A/B testing platforms or to make a decision. You want a tool that will suit you, not the tool with the best SEO.

You can also…

Use BuiltWith to Look At Other Brands in Your Space

This works nicely if you are in a niche industry. The oft-recommended platforms may not have integrations with your tech stack. Or they may not support experimentation to improve aspects and metrics that matter to you.

For example, there is a lot of conversion rate optimization in the Shopify space, but most brands accept that struggling with revenue tracking is part and parcel of setting up an A/B testing tool.

Do you want to struggle with anything that matters to your experimentation program?

That’s why the A/B testing tool you’re considering should have specific provisions to accommodate your testing needs. Like how Convert offers a custom Shopify app to get rid of the revenue tracking problem.

This may not yield the exact tool you need, but it will inform your consideration set.

To find what brands in your space are using, all you need is their website address to run a search on BuiltWith.

Or you can search the tools you’re considering and see who’s using them in your space. There are limits when you use a free account.

Tap Your Team

If you have a team that regularly runs tests in your organization, they’re one of the best to inform your consideration set. They’re already swimming in the pool so they’ll know the temperature.

Get their opinion. Ask them why they wish to go with specific tools. What features are non-negotiable and which are just nice-to-haves?

What past experience trumps fancy feature releases? The best vote of confidence is one that comes from a tester who has succeeded in the past with a particular platform.

Gather this information and structure it. If you find a recommendation, do a quick check against your problem to ensure that the suggestion has merit.

Build Your Consideration Set

At this stage, you’ve recognized 3 or 5 tools that you’re seriously interested in. That’s a solid starting point. But right now, your decision may be based on intangible factors, referrals, and brand recognition.

When you build your consideration set, you will create a matrix of these 3-5 A/B testing tools and hold them against the components of a problem. That is, how they measure up in solving the major problems you’re looking to fix with the right A/B testing tool for you.

Let’s illustrate:

PROBLEM: Delays in experiment deployment.

If you’ve pinpointed more than one essential problem, you’ll want to have another consideration set for those as well. But with the same tools, only different components of the problem.

The thing about multiple problems is not only that it gets complex to assess, but a surprising alternative can come out on top. That’s how you know you’re making objective decisions. Pretty neat, right?

Note: Since all problems have different levels of priority, ensure that you’ve weighted them accordingly.

So, how do you know what to fill into a consideration set like in the example above?

Filling in the Gaps

• Reviews
  
  Start with popular software review sites like G2, Capterra, and TrustRadius. Remember: Sponsored profiles may be displayed more. Yet, this has nothing to do with how good a tool is at solving your problems.

• Cons
  
  Aim for those 2-4 star reviews. The cons there are usually well thought out. Dig through them for issues that might be devastating to your goals. See something you’re trying to avoid? Put an “X” there on the consideration set.

Take critical reviews with a pinch of salt, though. What went sideways for someone else isn’t certain to go sideways for you too. But if you notice a common denominator in 3-4 reviews, then there’s a pattern worth paying attention to.

• Comparison Tables
  
  Some of the work is already done for you. Look at comparison tables — both from vendors and from unbiased 3rd party sources like HubSpot and CXL.

We have a tool to help you compare and contrast Convert Experiences, Google Optimize, Optimizely, VWO, AB Tasty, Qubit, and more.

Want to read CRO experts weigh in on how they buy A/B testing tools?

Here is what Ruben De Boer has to say about choosing an A/B testing tool:

First, of course, you have to make sure you have sufficient traffic for A/B testing, that at least a part of your company supports testing and that development has enough resources to implement winning experiments.

Next, take your current and future needs into account. Start by assessing the current situation, like your goals, needs, and who is involved. For example, if IT is fully involved, you could choose a server-side solution. Whereas if only marketers are involved, client-side is most likely the way to go.

Then, instead of only looking at the current situation, consider where you want to be in a few years. What needs do you have then? Will you scale up the number of experiments? Have more team members use the tool? Run different kinds of experiments? Discuss the possibilities to ramp up with sales reps of the various vendors.

Finally, conduct a free trial with the two-three tools that best match your current and future needs. During the free trial, check for the tool’s user-friendliness, (targeting) options, and other things applicable to your business (i.e., does it work with your single-page application).

Also, a personal connection is important, as you might need their customer support. For example, is there help with onboarding? If so, how happy are you with it? Also, contact their customer support. Besides checking the available channels (chat, mail, phone), see how quickly they respond (are they in the same time zone?) and how helpful they are. Finally, check with IT the tool’s impact on your site performance.

Ruben De Boer, Lead Conversion Manager at Online Dialogue

And this is Max Whiteside’s take. Max is the Content Lead at Breaking Muscle:

Define the testing scope, as doing so will enable you to restrict the testing capabilities of your A/B testing tool. Do you intend to conduct only A/B tests, or do you need a tool that can handle multi-page and multivariate testing as well?

Each tool operates in a unique manner and offers distinct test types. So, look for one that has all the features you need, so you don’t have to switch tools as your testing capacity grows.

Secondly, the budget A/B testing is an iterative process, which makes budget a crucial consideration when implementing a testing environment. You may need to repeat a test multiple times before achieving the desired result, at which point you can proceed to the next iteration. It necessitates both time and resources, and your budget will dictate its upper limits. Find a tool that won’t make a significant dent in your pocketbook.

Some testing tools may have a negative impact on the speed of your website while running a test. A slow-loading page can increase the bounce rate and irritate users, leading to fewer conversions. In such a scenario, neither the control nor the variation would perform optimally, rendering the A/B test results unreliable even if the minimum sample size is met.

Moreover, the flicker effect is minimal. The flicker effect occurs when a user briefly views the original web page before being redirected to the variant. Flicker can hinder user experience and skew test results. To maintain the reliability of your A/B tests, it is best to select a tool that does not produce a flickering effect. Determining whether the A/B testing tool slows down your website is therefore essential.

Max Whiteside, Content Lead at Breaking Muscle

Make Contact with the Tool

Up to this point you’ve been a secret admirer of the tools you’re considering. Now’s time to overcome your shyness and make contact.

And the best place to start is a free trial or demo.

Start a Free Trial & Book a Demo

You need to see the tool in action.

How does it perform in your various A/B testing applications? As a CRO for a Shopify (or any other ecommerce) store, for instance, you’ll want to know how it performs from testing CTA copy for pop-up offers to product image tests.

A demo shows you how the tool solves your problem while a free trial reveals if the platform backs up the claims with the goods (i.e. features, pricing, service, roadmap).

Note: Most enterprise-level A/B testing tools won’t get on a demo with you before they’ve pre-qualified you. And they do not offer free trials. This can be a blind spot in your evaluation process.

Gauge the Culture

You might ask, “Why does culture even matter in the tool I’m buying?”

When you use an A/B testing tool, the vendor becomes part of your experimentation culture. That relationship works out best if you’re similar in business practices and values.

Don’t worry, this won’t narrow down your search to where you only have 1 or 2 tools to consider. A good culture is actually not rare. You just want to avoid friction that can get too uncomfortable and prompt a tool change too soon.

So, how do you gauge an A/B testing tool’s culture? You can look at Glassdoor reviews. People who work or have worked there can give a fairly transparent (although anonymous) insight into the culture.

Another way to assess an A/B testing tool’s vendor is their interaction with the industry. Do they truly educate the space?

What type of content can you find on their blog and social media pages? How do other players in the industry interact with them?

via GIPHY

Are they looking beyond their tool to help you with improving your processes? It’s more than just posting “our tool is great” content. You can clearly spot a brand that’s invested in the success of its audience. It’s hard to hide such a bright light.

Request Information You Are Missing

Yes, you can ask the A/B testing tool vendor to fill in the gaps for you with Request for Information docs.

RFI documents are similar, so a template like this would work. What’s going to be unique in yours, besides the introductory information about your organization and the goals you want to achieve, are the questions you ask. You’ll want to know

• The vendor’s track record
• The success story of customers similar to your organization
• Pricing
• References, and more

But also importantly, get the name and contact info of the person responsible for answering the RFI.

Pre Purchase Gear Up

Part of the effort in making a successful A/B testing tool purchase is in the pre-purchase gear up — the transition from no tool or current tool to fresh software in your tech stack. What do you need to do to migrate smoothly?

First, reveal your pick to your team and get buy-in. If you’ve not been working alone so far, and you’ll use the new tool as a team, they need to be on the same page with you. Let them know the vendor you’ve picked. And show them why.

This is why having a structured approach really helps. Now you can show the meticulous considerations you went through to land on this decision so that it makes perfect sense to choose this tool. With everything done right thus far, buy-in is easy at this point.

Discuss your decision as a team. How does it connect with the current problems in your experimentation journey? How does it support your goals for growth? What are other users saying about the tool? What did you learn from your research? And how did it hold up in your consideration set?

Also, focus on the “why” behind choosing the vendor. Beyond the sheer capabilities of the tool, are you going to be working with a vendor that supports your testing goals? Do they provide the educational backing your team needs to make the most of the new A/B testing tool? What are the vendor’s goals for their users?

Then, draft up a timeline for the transition and prepare to roll out. Migrating needs to be planned carefully to prevent hiccups and costly delays.

This is why you’ll want to work with a vendor that supports you. Will the vendor assist with the migration? Can you shift your data to the new vendor? Is there a robust onboarding process? Find out what expertise you need to get that done and how long it’ll take for each stage.

Finally, zero in on your goals for the optimization program broken down by 3-6-9 months. That way, you have quarterly checkpoints to assess the decision you made. Is the chosen A/B testing tool and vendor keeping up with the expectations you started out with?

Careful here: Expectations need to be realistic. Make sure you don’t gauge your vendor on outcomes that they can’t influence.

Let’s make this an actionable checklist you can take with you.

A Handy Checklist of Factors to Keep In Mind As You Migrate

You want to migrate from good to better; smart to genius; amazing to blissful. That doesn’t happen by chance.

Tick all the checkboxes below so you can be sure the A/B testing tool you’re migrating to performs beyond your expectations and help you achieve your experimentation goals.

• Cost — Does the cost of the A/B testing tool justify the results you’re expecting? How does the cumulative investment in the tool measure up to your expected ROI?

• The Learning Curve — You don’t want to spend valuable experimentation hours trying to figure out how to do basic tasks with your chosen testing tool because of a UI cluttered with tons of features you’ll never use. Read reviews to learn about the learning curve of the tool.

• Flicker — Flicker occurs when users see the original version of a page flash on their screen before the A/B testing software presents them with the variant. This can mar the results of your test.
  
  Some tools like Convert Experiences have developed anti-flicker technology, so this doesn’t happen. But many free and open-source A/B testing tools don’t yet have that capability.

• Out of the Box Integrations — Your A/B testing tool isn’t a lone wolf. It has to team up with other tools in your tech stack to bring you maximum value. Check out the number of out-of-the-box integrations your CRO tool comes with. Does it integrate with vital services you need it to work with?

• Google Analytics — Ensure your tool has a smooth integration with Google Analytics. When setting up, this is one of the first tasks you’ll have to complete. Find out if you might need a developer to help with that tool.

• Goals — Goal setting is an integral part of your A/B testing. It adds context to your tests and illustrates the impact of your test results. How does your new tool handle this? Can you import goals from Google Analytics?

• Revenue Tracking — When migrating, you want to ensure your revenue tracking is properly configured for both one-time and multi-revenue events. Some tools require manual setup. You don’t want that.
  
  Convert Experiences automatically connects to Google Analytics Universal and Classic to pull values of transactions, revenue, and ordered items to inform the tests.

• A/A Test — Does the new tool pass A/A tests? If you have free trial access to an A/B testing platform, run this test. That’s how you know your tool is primed to handle A/B tests without biases — or not.

• Collaborator Settings — Particularly important if you work in a team. Can you control access to what your collaborators can view and make changes to?

And these 2 additional considerations to remember:

1. What Level of Customer Support Does the A/B Testing Tool Provide?

Another point of consideration in your assessment is the level of technical support available to you as a customer.

Buying an A/B testing tool may be an enormous commitment. You need to be sure that the tool provides enough technical support to your team in the event of difficulties.

The level of support offered by the top A/B testing tools on the market varies. With some research, you can find out if the testing tools you’re considering offer all or a combination of the following:

• Live support: offered by phone or video call
• Email support: offered by a dedicated email address.
• Live chat: using an inbuilt messaging app
• Documentation: an online repository of articles that provide solutions to common problems.
• Automated support: a set of pre-recorded responses.

Once you have confirmed the level of support available, you need to find out if the people stationed in the various support channels are CRO experts. It is especially important that you choose a tool with CRO experts for support personnel, as they can better provide the real-time help you need if you run into technical issues.

It is also worth noting that some A/B testing tools offer dedicated customer support as an add-on which you need to pay for or upgrade to a higher plan to access for free. Paying for dedicated technical support or upgrading to a plan that far outweighs your testing needs to access better support will increase the overall cost of the A/B testing tool.

At Convert, our team of developers and CRO experts provides technical support to our customers for free. Our support takes the form of live chat, online documentation, email support, and live support.

2. Does the A/B Testing Tool Impact Your Site Speed?

Another important consideration when buying A/B testing tools is their impact on website speed. Some A/B testing tools can add seconds to your website speed and increase load time. Slow website load time affects user experience and may affect your traffic.

Let’s get into the mechanism of how some of the top A/B testing tools on the market can impact your website speed. A/B testing tools usually fall into two categories:

1. Client-side A/B testing tools
2. Server-side A/B testing tools

Client-side A/B testing tools use a JavaScript snippet stored on a visitor’s browser to alter the content your server shows the visitor based on your targeting instructions. Server-side A/B testing tools deliver the exact content with alterations from the server to the user without using touching the user’s browser.

Server-side testing tools seem like the best option, but they can be expensive and not relevant for many use cases. You can delve more into the differences in our client-side vs server-side testing article.

Many of the top A/B testing tools out there are client-side testing tools. Client-side A/B testing tools load in two ways:

1. Synchronously

This is when the JavaScript snippet of your A/B testing software loads fully before your web page loads. This way of loading the scripts prevents flickering. But may cause your page speed to slow down, as the code snippet from your A/B testing solution load time adds to your overall page speed.

2. Asynchronously

This is when the code snippet from your A/B testing platform of choice and web page loads simultaneously. Tools that use this method of loading may be fast but are prone to flickering.

As I mentioned earlier, flickering occurs when the visitor sees the original content before the variation loads.

Convert is one of the fastest A/B testing software on the market. This is thanks to its Smart Insert Technology^TM which reduces its impact on your site speed.

4 A/B Testing Tools We Like

Where do you start in your search for the perfect A/B testing tool to buy? Right here with our top 5 favorites.

We’ve considered everything we talked about above and created a list based on what we know about our audience.

Let’s dive in!

1. Convert Experiences

2. Kameleoon

3. Optimizely

4. Google Optimize (and Optimize 360)

How to Buy a Google Optimize Alternative: Essentials You Need to Know

Since the Google Optimize sunset announcement, the marketing and optimization corner on LinkedIn has been in a frenzy. It seems no one is talking about anything else:

If you’ve been following this, you’ve probably drowned in the sea of noise the sunset has created. But here’s the thing with noise — it’s part misinformation, part truth. It’ll take some effort to sift through this.

The truth: Contrary to what some A/B testing tool vendors would like you to believe about their product, there is no best Google Optimize alternative. There is only the best one for you — it all depends on the CRO program you run and your portfolio of tests.

You will only find the best tool to switch to if you take an unbiased, data-based approach to choose from the myriad of A/B testing tools thrown in your face this period (instead of relying on other people’s opinions).

The misinformation: You’ve probably seen posts about Google announcing its preferred partners to replace Google Optimize. That is false.

Google said:

What Google is saying here is: When it comes to A/B testing side-by-side with Google Analytics, we’re working on an integration that’ll be as good as Optimize was. And these are the A/B testing providers we’re partnering with to test that integration. In a few months, we’ll make the API publicly available so ANYONE can integrate their A/B testing tools with GA4.

The top reason you see these 3 on the list is that they’re the easiest for Google to work with as they’re Google Cloud partners. That means their testing apps use Google cloud services instead of competitors like Amazon Web Services.

It isn’t nice for anyone to take advantage of a misinterpretation of this message to promote their product. But we’ve seen this a lot on LinkedIn.

In fact, if you examine your options (like we did when figuring out how we can show you the best way to buy a Google Optimize alternative)… Say, you search “Google Optimize alternative” or “Google Optimize alternatives”, you’ll see the SERP sentiment indicates something interesting.

Of the top 30 results for those keywords, Convert is mentioned 12 times, VWO is mentioned 19 times, and Optimizely is mentioned 15 times.

If you do the same on LinkedIn, for the top 20 Google Optimize alternatives posts, you’ll find Convert mentioned 10 times, VWO and Optimizely mentioned 17 and 14 times respectively.

You’ll find 6 tools that are mentioned almost everywhere you look:

• Convert
• VWO
• Optimizely
• AB Tasty
• Adobe Target
• Kameleoon

VWO, AB Tasty, and Optimizely are enterprise offerings that are Google Cloud partners. They are bound to get a ton of exposure. But watch out for the vendors who don’t have privileges but still see plenty of mentions. That is user satisfaction and word of mouth marketing talking!

Not Every Popular Alternative Is an A/B Testing Tool

How do you feel about a list of Google Optimize alternatives that includes just about any product as long as it has some type of A/B testing feature in it? I mean, it isn’t primarily an A/B testing tool but hey, there’s A/B testing inside… somewhere.

Beware of tools popping up overnight.

Your focus should be on migrating to a tool that specializes in A/B testing and won’t disappear (or deprecate a feature) on you. You most certainly don’t want to go platform shopping in a few months again.

Free Alternatives Can Come at a Cost

Many long-time users of Google Optimize are not used to paying for an A/B testing tool, especially if experimentation isn’t a strong focus in their organization. That’s fine. But if you were using GO seriously, try not to get caught up in the trap of free again.

If you’re opting for a free A/B testing tool, make sure:

• The plans aren’t severely hamstrung. You need some flexibility in terms of what you can test and what you can’t. Google Optimize was a free tool with decent capabilities. You need at least that. Don’t downgrade.
• More traffic doesn’t cost an arm and a leg. You don’t want to cap your optimization program at 50k or 100k monthly uniques. So, keep future costs in mind… you’ll have to expand at some point. That point shouldn’t trigger another A/B testing tool replacement search.

Free plans sprung up in the wake of Google Optimize sunset. These plans didn’t exist before now. Ask yourself why they’re really available now and read between the lines of the offers.

Use this Step-by-Step Framework to Find Your Ideal A/B Testing Tool

There’s actually a smart technique to shop for the Google Optimize alternative you’ll always be happy with and will be beneficial to your experimentation program in the long run (cost and everything else considered).

That’s what we’re talking about here. Take your budget, traffic volume, experimentation maturity level, etc., into consideration and make a choice you’d love.

1. Leave Google Optimize Immediately

Migration takes time, especially in the case of Google Optimize for A/B testing + Google Analytics for reporting. You may also have to contend with getting set up on GA4 (coming from UA). Because waiting till September 30th is going to be a hair-pulling regretful situation.

Since UA is sunsetting on 1st July 2023, it means you have to act faster than everyone else. Find your alternative today.

And we’re here to guide you through. The next thing you want to work on is…

2. Structure Your A/B Testing Program

If you didn’t have a structure before now, GO’s sunset is an excellent wake-up call.

My recommendation to those impacted by this sunset is to take this as an opportunity to learn and rethink their overall testing strategy.

It won’t only allow them to pick the right tool to migrate to, but could also make them realize that either A/B testing might not be a good fit at the moment, or that they’re leaving money on the table by not doing it more actively.

Carlos Trujillo, Experimentation Strategist at Speero

• Why are you testing?
• What benefits does it bring to your business?
• What’s your website’s monthly traffic?
• What’s your testing volume?
• What do you need from your ideal A/B testing tool?
• What do you want to achieve from your A/B testing program in the next 6 months, 9 months, to 1 year?
• What has your testing strategy in the past year been?
• Do you use a CRO agency or in-house talent?

It is important to answer these questions before going shopping for a replacement. This will help you have a hard-copy idea of what your testing program looks like, your maturity level, your specific needs and use cases, and what A/B testing means to your organization (so you can decide how much of your budget makes sense to dedicate to a paid tool).

Make Maslow proud: Take stock of your current A/B testing, multivariate and optimisation needs. Which features do you love? Which functions do you loathe? What are you ultimately looking for in your new platform partner?

Miles Hoogwerf, CRO Consultant at DEPT®

Try to extrapolate everything you do to a 1-year view in case the tools you shortlist are the ones that only offer yearly plans.

3. Use Vendor Comparison Tables

When you’re done with step 2, you should know:

• What A/B testing tool features matter most to you
• How much you can pay for an A/B testing tool
• The monthly traffic range that’s suitable for your organization today and in the near future
• The technical capabilities of your team
• The volume of tests you run on a monthly basis
• Reporting tools you need, etc.

Check out Convert’s tool comparison guide, or find out how Convert compares with VWO or Optimizely.

You can also use G2 or TrustRadius to get a fairly accurate comparison to kickstart your research.

You can also drill further down by talking to people. Chat with people in your industry, read reviews, or engage with CRO experts. Ask them about their experience using the tools they use and listen to unbiased feedback only.

4. Consider Integrations, Privacy and Security

Those looking to move to a Google Optimize replacement and not have to move again anytime soon should pay particular attention to integrations. Why? Because integrations with other platforms and tools are key to keeping your optimization program running smoothly. By integrating the right services, you will be able to maximize your work without having to worry about compatibility issues.

Google Optimize was very popular so almost any tool you had could integrate with it. Your next step should be a tool with almost as much integration.

Using Zapier doesn’t count. You want something that plays well with other tools without stressing you out to build a custom workflow.

In the coming years, privacy regulations will impact how well A/B testing tools work. Find a tool that’s at the forefront of privacy and positioned to conquer in a cookieless future — like Convert Experiences. It’s the same for security.

When you plan to commit to testing for the long run, these are crucial to your goal.

5. Your Search Isn’t Complete Without Free Trials and Demos

Sign up for free trials and prepare your questions for demos. This will take a lot of time but that’s nothing compared to the time it’ll take to migrate again if you make the wrong choice.

6. Make an Objective Decision

By now, you should have a spreadsheet filled with the pros and cons of each tool. Award points to pros and subtract points for cons, according to your own formulated weighted average. This is all for you and your unique use case, remember?

At the end of your columns for each tool, you should have the numbers there pointing you toward the best option to switch to. Don’t lean towards the noise you’ve been hearing. If your research, free trials, demos, feedback, reviews, and comparison tables point you towards a specific tool (without a shadow of a doubt), that’s your tool.

Key Takeaways

Finding the right A/B testing software for your needs isn’t as easy as it seems. Complex factors like your testing needs, pricing, integrations, available skillset, set-up, training, and more, need to be taken into consideration.

When you start with the problem, ask the right people, build your consideration list, and build a consideration set based on a careful analysis of the crucial factors that can make or break your experience with your A/B testing tool, you’re already halfway there.

You can choose to start your test with the tools we’ve listed above. But if that’s not your cup of tea, then we’ve given you all you need to go out there, research over 50 A/B testing tools, and find the right one for your unique needs.

Because the ultimate decision lies with you.