Best Free and Open Source A/B Testing Software
January 13, 2021 –
There is something special about the word FREE.
It conjures excitement, giddiness, and desire.
But does it have the same appeal when used to describe A/B testing tools?
Free A/B testing tools? (Do you hear the ring? Do you see conversions just rolling in?)
In this article, we will check out a few free A/B testing software that can start you on your experimentation journey.
Read more to find out:
- The power of using free tools,
- The top free A/B testing tools for 2021,
- The inherent but invisible dangers of using free tools.
Before we tackle the points above, let’s look at the power of A/B testing…
The Power of A/B Testing Software
Whether paid, free, or open-source, A/B testing software allows you to systematize experimentation with immediate results and analytics at your fingertips.
Increased conversions mean a boost in your bottom line. Experimentation software easily tracks and measures changes. For data-driven enterprises that depend on websites to woo their customers into taking action, A/B testing is a must.
When you look at price ranges, which goes from thousands per month to free, it might tempt you to think dollars and cents are the only reasons someone may choose a free tool. But the truth is, there are many reasons someone may use a free A/B testing tool. Let’s look at a few…
Why Use A Free A/B Testing Tool?
There are a couple of reasons you may opt to use a free A/B testing tool. They include:
- You’re new to experimentation.
You don’t know enough yet to invest in a high-end tool. You want to play around with a free A/B testing tool to understand the testing theory.
You want to use a free tool as your playground. Then implement your learnings without committing to the hefty price tags that are sometimes associated with paid A/B testing tools.
- You need proof.
This is a common scenario. You want to convince the C-suite and other stakeholders that A/B testing should be part of your growth strategy but need proof. Without a budget to create that proof, you can illustrate the potential power of testing with a free tool.
- You’re looking to learn and grow.
Similar to the first entry on this list, if you are new to CRO and experimentation concepts, using free tools is a great way to learn and bulk up your skills.
Limitations of Free A/B Testing Software
Most free CRO tools and A/B testing platforms have inherent limits. Even the most functional tools usually lack many of the beloved features that come standard with paid tools.
An often overlooked component of free tools that make them subpar to paid tools is their lack of customer support. Customer support can be valuable when using a SaaS tool. Especially if you are developing specialized testing codes or need specific integrations. The help of a knowledgeable support team can be the difference between using a tool to get an average lift, versus, mouth-dropping lifts.
This point is something that the SaaS industry doesn’t like to bring up. But it is part of the development cycle of software — bugs.
Even the best software can experience issues with bugs. But with free software, it can be even worse. They may not have dedicated teams who can quickly address issues. And if you are testing and you experience a bug, you don’t want to have to wait weeks before it is fixed. Or worse, if there is no customer support, you may not even be able to confirm they are aware of the bug and are working on it.
Josh Brown of HelpJuice outlines a few more limitations of free A/B testing software:
The limits of free tools as compared to paid tools typically fall under one of the following:
↪️ Features are limited. Either you don’t get access to all the tool’s features or your usage of a feature is capped.
↪️ Free until a certain threshold (typically traffic) is hit. For example, a tool may be free for under 5,000 visits per month, once you surpass this you must move to a paid plan.
↪️ Free for only a limited time.
So now that you know the limitations of free A/B tools, let’s dive into how you choose the best free tool.
How to Choose A Free A/B Testing Tool
Choosing a free or open-source A/B testing tool is not as challenging as with other software like email management systems, or productivity tools, there are just not as many options.
Here a few suggestions to remember when picking a free experimentation tool:
- Analyze your needs.
The clearer you are, the easier it will be to make a good choice. - Know your numbers.
Many free tools are really freemiums that have limits on the number of tests you can run, and traffic your site receives. Knowing these numbers in advance, you can determine if you fall within the parameters. - Clarify your goals.
What are you trying to get out of the tool? What do you want to accomplish? Will the functionality of the tool allow you to meet your goal?
Josh Brown also offers his advice on how to choose a free A/B testing tool. He says:
Now that you have an overview of free A/B testing software, let’s look at a few specific tools.
Best Free A/B Testing Tools
When researching free A/B testing tools in the market, it was surprising how tiny the number was. Below is a review of what we found…
1. Google Optimize
Google Optimize, formerly Google Analytics Content Experiment, major benefits is its natural integration with Google Analytics. For a free tool, Google Optimize has several features, including:
- A/B testing
- Multivariate testing (MVT)
- A variety of targeting options, including Javascript and Geo-targeting.
Benefits
Seamless integration with other Google products.
Nate Nead of SEO.co touts the efficiency of Google Optimize:
The product is free for up to five users, integrates seamlessly with your Google Analytics account and includes advanced modeling tools that allow users to test in multivariate formats. While there is no plugin for major content management systems like WordPress, implementation is straightforward with some simple code insertions. We have used the paid version as accounts have scaled beyond the five initial free licenses. However, the paid version is well worth the cost.
As hefty as this tool is, there are some limits…
Limits
As touted as Google Optimized is, there are still plagued with limits that don’t exist in paid tools like Convert.
Multivariate testing (MVT) options are limited (a standard option on the Convert platform).
Google Optimize also doesn’t offer “multi-armed bandit dynamic weight adjustment”.
The best way to check out a tool is by exploring it. Get full access to Convert for 15-days for free. A/B test, check out all our integrations and see why so many optimizers chose Convert as an Optimizely alternative.
Brian O’Sullivan first started running A/B tests at Google before optimizing at Dropbox. He’s now a SaaS growth consultant who starts his clients on Google Optimize. But he is also clear of Google Optimize’s limits:
As they get results and start buying into A/B testing, Google Optimize can start holding them back. The biggest issue is the limits. You can only run 5 tests at the same time, goals are capped at 3 per test, and there’s a limit on the code you can insert into their code editor (10240 characters). On top of that analyzing test segments after they’ve run in GA is cumbersome to set up.
As clients start ramping up their A/B testing and want to run more than 5 tests at once I encourage them to move to a paid tool.
Even with these limits, Google Optimize was a fan favorite for free A/B testing tools.
Next on the list is a tool called Nelio.
2. Nelio
Nelio is an open-source A/B testing software for WordPress websites. Nelio allows you to change your WordPress website in an A/B testing environment.
A freemium tool, there is no cost for Nelio if you have a small amount of traffic coming to your site.
Benefits
As a WordPress plugin, getting Nelio set up is not difficult and has solid reviews.
Limits
Limited to sites built on WordPress. Per a Kinsta article, Wild and Interesting WordPress Statistics and Facts (2020), there are over 20 billion WordPress pages. Wow!
But, even with that phenomenal number, there are still millions of websites that are built on other platforms. This means using Nelio to A/B test is not an option for them.
Also remember, this is a Freemium plugin.
3. Vanity
Vanity is another free and open-source platform that describes itself as “an Experiment Driven Development framework for Rails.”
If you are not familiar with Rails, it’s a Ruby (programming language) web application development framework.
Vanity allows users to measure multiple metrics through A/B testing.
Benefits
If you are familiar with Rails, and use this as your development platform, having access to this open-source tool that allows you to run small A/B tests is handy.
Limits
However, this software has limited reach. It does not allow you to test full websites, run concurrent tests, or a host of other functions you would expect in a full A/B testing tool.
3. GTM Testing
Again, Google Tag Manager is not a true A/B testing tool that allows you to test and measure changes to your entire website. However, you can perform a handful of useful A/B tests in GTM.
They describe their process as follows:
Benefits
One of the number one benefits of running a test through GTM is it is cost effective.
Limits
A/B testing using GTM is not what most users would classify as “easy.” If you are a newbie or new to CRO, this may not be an easy tool to jump into.
Bonus: HotJar
If you are entrenched in CRO and experimentation, you’re probably yelling at me right now, “Hotjar is NOT an A/B testing tool.” You are 100% correct. Hotjar is a heat mapping tool that provides feedback on how visitors behave on your website.
So why is it on our list of free A/B testing tools?
When we reached out to the CRO community to get their input on free and open-sourced A/B testing tools, it kept showing up. So we added it here as a “bonus.”
Hotjar is a freemium tool that allows you to see and analyze the behavior of your website visitors.
Benefits
Hotjar gives you qualitative data that you don’t get from qualitative-based experimentation tools. Tools like Hotjar actually work well alongside a proper A/B testing tool.
Limits
In this context, the main limit of Hotjar is that it is not a true A/B testing tool.
Free A/B Testing Tools Housed In Other Software Don’t Equal Full Standalone Tools
Many free CRO tools are misclassified as A/B testing software because they are used in conversion optimization.
An example of this is Five Second Test, which allows you to show your landing page to people for five seconds before being asked a series of questions.
Other examples include:
- SES Theme Split Test
With this plugin, you can test out different WordPress templates and themes. - JustUno
Want to specifically A/B test your pop-ups? Josh Brown of HelpJuice suggested this tool. - Mailchimp
The Experts’ Tips and Tricks on Using Free A/B Testing Software
We asked a handful of CRO experts to share their favorite free A/B testing tools and at what point they would move to a paid platform.
Below are some tips they shared:
Raul Mercado, of Camping Helper, shares what pushes him to purchase a full-bodied A/B testing tool:
Ottomatias Peura, the CMO of Speechly, uses A/B testing as part of their conversion strategy. He shares the following:
Bruce Hogan spent the past 10 years in digital marketing and is the Co-Founder & CEO of SoftwarePundit, an internet business that helps SMBs adopt the best software for their needs. He’s used a host of CRO solutions and offered this advice:
Faizan Fahim, a digital marketer at ServerGuy who oversees all the digital content of the site, shares his thoughts:
Trevor Ford, currently Head of Growth at Yotta Savings, explains the limits of free A/B testing and when migrating to paid ones makes sense:
Main Takeaway: Great Free Tools Available But You Get What You Pay For
Holly Winters of BrandCave encapsulated it best:
This doesn’t even factor in the level of additional services that are included in software packages that are sometimes overlooked, like customer support, branded CRO communities you can tap into, learning portals hosted by paid tools to help you become a better tester and CRO practitioner.
Lastly, a paid A/B testing platform like Convert Experiences can increase your conversion rates and your bottom line. Experimentation tools like Convert Experiences give you a clear ROI on your plan investment. When used to its full potential, the tool can pay for itself.
The best way to check out a tool is by exploring it. Get full access to Convert for 15-days for free. A/B test, check out all our integrations and see why so many optimizers chose Convert as an Optimizely alternative.
Can someone say, “Paid equals better than free?”
There is a reason medium size and enterprise companies invest in paid tools. They know it is a strategic investment that will give them a return.
The Bottom Line…
Holly Winters said it best, “You get what you pay for.” While free, freemium, and open-source tools have their place, they can’t 100% reach the robust functionality of a paid tool.
So, if you are a business new to the experimentation world, and have small numbers of visitors coming to your site, free A/B testing tools are fine. But as you grow, you should consider using a tool like Convert Experiences that gives you a seamless experimentation platform that can support your optimization efforts and push for growth.
What is your experience with free A/B testing tools versus paid?
