How to Buy an A/B Testing Tool
Did you know there are over 50 A/B testing tools on the market?
Yes, there are that many.
With so many A/B testing tools around, how do you know which one to choose?
The best A/B testing software isn’t necessarily the most expensive or the most popular. The testing platform that matches your optimization needs, budget and is easy to set up will be the best A/B testing software choice for you.
How do you narrow down ‘the one’ from all the top A/B testing tools on the market?
Let us help you out!
Do You Have Enough Traffic to Test?
A/B testing doesn’t work for low-traffic websites. Before you begin testing, your website must have enough traffic to send to each variation of your A/B test. Without adequate traffic, your experiments will not reach statistical significance and may run for a long time.
In a typical A/B test, you need to drive thousands of visitors to each variation. At Convert, we recommend sending at least 5K visitors to each variation and around 1000 conversions to produce conclusive results of which variant is the clear winner.
Head over to Convert’s A/B testing calculator to find out how much traffic, conversions, and average order per revenue you need to run your A/B test. In the event you lack the required traffic to A/B test, head over to our guide on testing with low traffic.
What are Your Testing Needs?
Your immediate experimentation needs and future growth will factor into your A/B testing tools comparison. For example, you may start out doing just A/B testing and transition into using other features of your testing tools like personalizations and multivariate testing as your website and optimization program grows.
Your optimization needs will determine the features you need in an A/B testing tool. There are non-negotiable features that everyone needs to run experiments like advanced audience targeting, rich reporting, code/visual editor, post-segmentation and more.
Evaluate your needs against the features the tool offers. With software, more features bring complexity you may not necessarily need and drive up the cost. For example, if you want to run A/B tests, a tool offering multi-armed bandit test features would not be right for this. This feature is nice to have, but it will drive up the cost of the tool and you may not even get any use out of it.
What Level of Customer Support Does the A/B Testing Tool Provide?
Another point of consideration in your assessment is the level of technical support available to you as a customer.
Buying an A/B testing tool may be an enormous commitment. You need to be sure that the tool provides enough technical support to your team in the event of difficulties.
The level of support offered by the top A/B testing tools on the market varies. With some research, you can find out if the testing tools you’re considering offer all or a combination of the following:
- Live support: offered by phone or video call.
- Email support: offered by a dedicated email address.
- Live chat: using an inbuilt messaging app.
- Documentation: an online repository of articles that provide solutions to common problems.
- Automated support: a set of pre-recorded responses.
Once you have confirmed the level of support available, you need to find out if the people manning the various support channels are CRO experts. It is especially important that you choose a tool with CRO experts for support personnel as they can better provide the help you need if you run into technical issues.
It is also worth noting that some A/B testing tools offer dedicated customer support as an add-on which you need to pay for or upgrade to a higher plan to access for free. Paying for dedicated technical support or upgrading to a plan that far outweighs your testing needs to access better support will increase the overall cost of the A/B testing tool.
At Convert Experiences, our team of developers and CRO experts provides technical support to our customers for free. Our support takes the form of live chat, online documentation, email support, and live support.
Does the A/B Testing Tool Impact Your Site Speed?
Another important consideration when buying A/B testing tools is their impact on website speed. Some A/B testing tools can add seconds to your website speed and increase load time. Slow website load time affects user experience and may affect your traffic.
Let’s get into the mechanism of how some of the top A/B testing tools on the market can impact your website speed. A/B testing tools usually fall into two categories:
- Client-side A/B testing tools
- Server-side A/B testing tools
Client-side A/B testing tools use a JavaScript snippet stored on a visitor’s browser to alter the content your server shows the visitor based on your targeting instructions. Server-side A/B testing tools deliver the exact content with alterations from the server to the user without using touching the user’s browser.
Server-side testing tools seem like the best option, but they can be expensive and not relevant for many use cases. You can delve more into the differences in our Client-side vs Server-side testing article.
Many of the top A/B testing tools out there are client-side testing tools. Client-side A/B testing tools load in two ways:
1. Synchronously
This is when the JavaScript snippet of your A/B testing software loads fully before your web page loads. This way of loading the scripts prevents flickering. But may cause your page speed to slow down, as the code snippet from your A/B testing solution load time adds to your overall page speed.
2. Asynchronously
This is when the code snippet from your A/B testing platform of choice and web page loads simultaneously. Tools that use this method of loading may be fast but are prone to flickering. Flickering occurs when the visitor sees the original content before the variation loads.
Convert is one of the fastest A/B testing software on the market. This is thanks to its Smart Insert TechnologyTM that reduces its impact on your site speed.
The best way to check out a tool is by exploring it. Get full access to Convert for 15-days for free. A/B test, check out all our integrations and see why so many optimizers chose Convert Experiences as an Optimizely alternative.
What Other Tools Do You Need To Integrate the A/B Testing Software With?
Your A/B testing tool is just one piece in your optimization tech stack. Other software like heatmap tools and analytics tools are important to gather qualitative and quantitative data that will inform hypotheses that fuel experiments.
There is a temptation to spring for A/B testing software that offers these different functionalities in one tool. Makes the job easier, right?! Not always!
These all-in-one types of A/B testing tools are often bloated and complex to use. Using complex and bloated A/B testing solutions has led to the widespread belief among marketers that A/B testing tools are difficult to use.
Did you know companies who used all-in-one tools spent $1.4 million more than those who used software tailored to their specific needs? Why spend more when you can achieve better results with less!
Another strike against all-in-one CRO tools is that their focus area is spread too wide and they may not offer the best when compared with specialized tools.
Instead of an all-in-one software touted as one of the top A/B testing tools around, you are better off buying a specialized A/B testing software that focuses solely on experimentation like Convert Experiences. And choosing specialized web and behavioral analytics tools like Heap Analytics and Hotjar.
This brings us to integrations. It is not enough to buy an experimentation-focused A/B testing platform; buy one that will integrate with other tools in your optimization stack.
As an experimentation-focused tool, Convert Experiences offers over 100 integrations with behavioral analytics, web analytics, ecommerce, call tracking, and more software that makes up a robust optimization stack.
Is Training and A/B Testing Solution Implementation Available?
One of the core aspects of your A/B testing tools assessment is software and implementation training.
Apart from being easy to set up, does the software you’re considering offer onboarding, training on how to use their tool, implement their tracking snippet?
Is the A/B testing solution offering this training as a part of your plan? Will you have to pay extra to get training on how to use their tools?
This is an important consideration as many A/B testing tools offer basic onboarding as part of your plan and additional training as an add-on that you need to pay for. Optimizely alternatives like Convert Experiences offer onboarding, basic training, and developer orientation to every customer for free.
Book a demo with Convert and see how testing can be more accurate and more profitable.
Book Demo
What Skills Do You Need to Have to Use the A/B Testing Tool?
The more intricate the A/B testing tool, the more sophisticated skills you need to develop to use it.
Does your team have a front-end developer? A dedicated developer may be necessary for coding complex tests and sometimes, implementing and setting up the tool. How good is your optimization team with statistics?
Evaluate the resources and skills that will be needed to use the A/B testing software you’re considering. Some tools will require more investment in technical resources than what you’re envisioning.
Remember, more features often bring a complexity that you may not have time or resources for. Since the goal is to increase conversions and revenue, an A/B testing software that is inclusive of different skill levels and easy to set up, like Convert Experiences, is the best bet for you.
How Much Does the Tool Cost?
Pricing is of course an important factor to consider. But the pricing of your A/B testing tool isn’t limited to just having access to the software itself. There are so many facets related to pricing that it will be remiss not to cover them here.
One of the biggest issues in the optimization industry is transparent pricing. Many top A/B testing tools gate their pricing and require you to call their sales team for a quote. Others are notorious for changing pricing models without notice. This makes it difficult to compare pricing across tools. At Convert Experiences, we have transparent pricing on our website that visitors can access without contacting our sales team.
Flexible Pricing
Some A/B testing platforms may not offer flexible billing and require a year’s commitment upfront. This may mean extra costs, per their exit clause, in the event you have to leave your contract with the A/B testing platform early. A tool that offers flexible billing month-to-month may be better in the long run for your business.
Another consideration point is the cost of going over your monthly/yearly allotted tested visitors. You need to find out how much this costs. It may be better to move up to the next plan if your website traffic is close to the threshold of allotted testing visitors.
Extra Costs
Another cost you may need to factor into the overall yearly tool pricing is dedicated support and training add-ons. Some testing tools require payment for additional training outside onboarding and dedicated customer support. Convert Experiences offers training and technical support to all customers on all our plans at no extra cost.
Also, factor in the cost of internal resources like developers, UX experts, statisticians, and other members of your optimization program to the total yearly cost of your testing tool.
Finally, we come to pricing stability.
It is not enough to buy one of the top A/B testing tools on the market. You need a CRO tool whose pricing is not subject to unexpected hikes. Unexpected price hikes may leave your optimization program in the lurch. For example, Optimizely has gone through sudden pricing changes in the past that left many customers unable to continue using the platform. Its recent acquisition by Episerver might bring more pricing changes.
At Convert, we respect grandfathering and do not force customers to move to higher plans as our pricing changes. Here is a nice A/B testing tools comparison of Convert and Optimizely.
Conclusion
Finding the right A/B testing software for your needs isn’t as easy as it seems. Complex factors like your testing needs, pricing, integrations, available skillset, set-up, training, and more, need to be taken into consideration. Ultimately, the decision lies with you…
