5 Ways Digital Marketers Use Call Tracking to Grow Their Business
Blog /A/B Testing

Best Free and Open Source A/B Testing Software

13th Jan 2021 – Free A-B testing tools

There is something special about the word FREE. 

It conjures excitement, giddiness, and desire.

But does it have the same appeal when used to describe A/B testing tools? 

Free A/B testing tools? (Do you hear the ring? Do you see conversions just rolling in?)

In this article, we will check out a few free A/B testing software that can start you on your experimentation journey. 

Read more to find out:

  • The power of using free tools, 
  • The top free A/B testing tools for 2021, 
  • The inherent but invisible dangers of using free tools.

Before we tackle the points above, let’s look at the power of A/B testing…

The Power of A/B Testing Software

Whether paid, free, or open-source, A/B testing software allows you to systematize experimentation with immediate results and analytics at your fingertips.  

Increased conversions mean a boost in your bottom line. Experimentation software easily tracks and measures changes. For data-driven enterprises that depend on websites to woo their customers into taking action, A/B testing is a must.

When you look at price ranges, which goes from thousands per month to free, it might tempt you to think dollars and cents are the only reasons someone may choose a free tool. But the truth is, there are many reasons someone may use a free A/B testing tool. Let’s look at a few…

Why Use A Free A/B Testing Tool?

Free A/B testing tools

There are a couple of reasons you may opt to use a free A/B testing tool. They include: 

  • You’re new to experimentation.

You don’t know enough yet to invest in a high-end tool. You want to play around with a free A/B testing tool to understand the testing theory.

You want to use a free tool as your playground. Then implement your learnings without committing to the hefty price tags that are sometimes associated with paid A/B testing tools.

  • You need proof.

This is a common scenario. You want to convince the C-suite and other stakeholders that A/B testing should be part of your growth strategy but need proof. Without a budget to create that proof, you can illustrate the potential power of testing with a free tool.

  • You’re looking to learn and grow.

Similar to the first entry on this list, if you are new to CRO and experimentation concepts, using free tools is a great way to learn and bulk up your skills. 

Limitations of Free A/B Testing Software

Most free CRO tools and A/B testing platforms have inherent limits. Even the most functional tools usually lack many of the beloved features that come standard with paid tools. 

An often overlooked component of free tools that make them subpar to paid tools is their lack of customer support. Customer support can be valuable when using a SaaS tool. Especially if you are developing specialized testing codes or need specific integrations. The help of a knowledgeable support team can be the difference between using a tool to get an average lift, versus, mouth-dropping lifts.  

This point is something that the SaaS industry doesn’t like to bring up. But it is part of the development cycle of software — bugs.

Even the best software can experience issues with bugs. But with free software, it can be even worse. They may not have dedicated teams who can quickly address issues. And if you are testing and you experience a bug, you don’t want to have to wait weeks before it is fixed. Or worse, if there is no customer support, you may not even be able to confirm they are aware of the bug and are working on it. 

Josh Brown of HelpJuice outlines a few more limitations of free A/B testing software:

The limits of free tools as compared to paid tools typically fall under one of the following:

↪️ Features are limited. Either you don’t get access to all the tool’s features or your usage of a feature is capped.

↪️ Free until a certain threshold (typically traffic) is hit. For example, a tool may be free for under 5,000 visits per month, once you surpass this you must move to a paid plan.

↪️ Free for only a limited time.

So now that you know the limitations of free A/B tools, let’s dive into how you choose the best free tool.

How to Choose A Free A/B Testing Tool

Choosing a free or open-source A/B testing tool is not as challenging as with other software like email management systems, or productivity tools, there are just not as many options. 

Here a few suggestions to remember when picking a free experimentation tool:

  • Analyze your needs.
    The clearer you are, the easier it will be to make a good choice.
  • Know your numbers.
    Many free tools are really freemiums that have limits on the number of tests you can run, and traffic your site receives. Knowing these numbers in advance, you can determine if you fall within the parameters.
  • Clarify your goals.
    What are you trying to get out of the tool? What do you want to accomplish? Will the functionality of the tool allow you to meet your goal?

Josh Brown also offers his advice on how to choose a free A/B testing tool. He says:

Before making a decision on CRO tools, make sure it actually makes sense to use one (even if it is free). 

Figure out which tests can move the needle the most (as there are many types of tests that can be done). By Josh Brown

Now that you have an overview of free A/B testing software, let’s look at a few specific tools.

Best Free A/B Testing Tools

When researching free A/B testing tools in the market, it was surprising how tiny the number was. Below is a review of what we found… 

1. Google Optimize

Free A/B Testing Tool - Convert as an alternative to Google Optimize

Google Optimize, formerly Google Analytics Content Experiment, major benefits is its natural integration with Google Analytics. For a free tool, Google Optimize has several features, including:

  • A/B testing
  • Multivariate testing (MVT) 
  • A variety of targeting options, including Javascript and Geo-targeting. 


Seamless integration with other Google products. 

Nate Nead of SEO.co touts the efficiency of Google Optimize:

The product is free for up to five users, integrates seamlessly with your Google Analytics account and includes advanced modeling tools that allow users to test in multivariate formats. While there is no plugin for major content management systems like WordPress, implementation is straightforward with some simple code insertions. We have used the paid version as accounts have scaled beyond the five initial free licenses. However, the paid version is well worth the cost. 

As hefty as this tool is, there are some limits…


As touted as Google Optimized is, there are still plagued with limits that don’t exist in paid tools like Convert. 

Multivariate testing (MVT) options are limited (a standard option on the Convert platform).

Google Optimize also doesn’t offer “multi-armed bandit dynamic weight adjustment”. 

The best way to check out a tool is by exploring it. Get full access to Convert for 15-days for free. A/B test, check out all our integrations and see why so many optimizers chose Convert as an Optimizely alternative.

Brian O’Sullivan first started running A/B tests at Google before optimizing at Dropbox. He’s now a SaaS growth consultant who starts his clients on Google Optimize. But he is also clear of Google Optimize’s limits:

As they get results and start buying into A/B testing, Google Optimize can start holding them back. The biggest issue is the limits. You can only run 5 tests at the same time, goals are capped at 3 per test, and there’s a limit on the code you can insert into their code editor (10240 characters). On top of that analyzing test segments after they’ve run in GA is cumbersome to set up.

As clients start ramping up their A/B testing and want to run more than 5 tests at once I encourage them to move to a paid tool.

Even with these limits, Google Optimize was a fan favorite for free A/B testing tools.

Free A/B Testing Tools

Next on the list is a tool called Nelio.

2. Nelio

Free A/B testing tool

Nelio is an open-source A/B testing software for WordPress websites. Nelio allows you to change your WordPress website in an A/B testing environment.   

A freemium tool, there is no cost for Nelio if you have a small amount of traffic coming to your site. 


As a WordPress plugin, getting Nelio set up is not difficult and has solid reviews.

Free A/B Testing Software - Reviews of A/B testing that can be used on WordPress


Limited to sites built on WordPress. Per a Kinsta article, Wild and Interesting WordPress Statistics and Facts (2020), there are over 20 billion WordPress pages. Wow! 

But, even with that phenomenal number, there are still millions of websites that are built on other platforms. This means using Nelio to A/B test is not an option for them. 

Also remember, this is a Freemium plugin. 

3. Vanity

Free A/B Testing Software

Vanity is another free and open-source platform that describes itself as “an Experiment Driven Development framework for Rails.”

If you are not familiar with Rails, it’s a Ruby (programming language) web application development framework.  

Vanity allows users to measure multiple metrics through A/B testing.



If you are familiar with Rails, and use this as your development platform, having access to this open-source tool that allows you to run small A/B tests is handy. 


However, this software has limited reach. It does not allow you to test full websites, run concurrent tests, or a host of other functions you would expect in a full A/B testing tool.

3. GTM Testing

Free and Open-Source A/B Testing Tools

Again, Google Tag Manager is not a true A/B testing tool that allows you to test and measure changes to your entire website. However, you can perform a handful of useful A/B tests in GTM. 

They describe their process as follows: 

Free A/B Testing Tools. How does GTM Testing Work.


One of the number one benefits of running a test through GTM is it is cost effective.


A/B testing using GTM is not what most users would classify as “easy.” If you are a newbie or new to CRO, this may not be an easy tool to jump into. 

Bonus: HotJar

Free A/B Testing Tools

If you are entrenched in CRO and experimentation, you’re probably yelling at me right now, “Hotjar is NOT an A/B testing tool.” You are 100% correct. Hotjar is a heat mapping tool that provides feedback on how visitors behave on your website. 

So why is it on our list of free A/B testing tools?

When we reached out to the CRO community to get their input on free and open-sourced A/B testing tools, it kept showing up. So we added it here as a “bonus.”

Hotjar is a freemium tool that allows you to see and analyze the behavior of your website visitors.


Hotjar gives you qualitative data that you don’t get from qualitative-based experimentation tools. Tools like Hotjar actually work well alongside a proper A/B testing tool. 


In this context, the main limit of Hotjar is that it is not a true A/B testing tool.

Free A/B Testing Tools Housed In Other Software Don’t Equal Full Standalone Tools

Many free CRO tools are misclassified as A/B testing software because they are used in conversion optimization. 

An example of this is Five Second Test, which allows you to show your landing page to people for five seconds before being asked a series of questions.

Other examples include:

  1. SES Theme Split Test 
    With this plugin, you can test out different WordPress templates and themes.
  2. JustUno 
    Want to specifically A/B test your pop-ups? Josh Brown of HelpJuice suggested this tool.
  3. Mailchimp

The Experts’ Tips and Tricks on Using Free A/B Testing Software

We asked a handful of CRO experts to share their favorite free A/B testing tools and at what point they would move to a paid platform.

Below are some tips they shared:

Raul Mercado, of Camping Helper, shares what pushes him to purchase a full-bodied A/B testing tool:

Free A/B Tesitng Tools. I would consider purchasing a tool for A/B testing if I needed to perform more complicated A/B tests based on user segmentation like device, age, country, sex, and so on. Free options will not usually give you these kinds of specifics...Raul Mercado

Ottomatias Peura, the CMO of Speechly, uses A/B testing as part of their conversion strategy. He shares the following:

Free A/B Testing Tools:  We use paid tools to run most of our A/B tests on our site, which seems to work well for us. The main difference between free tools and paid alternatives is in the extent you can do multivariate testing and goal setting, Ottomatias Peura

Bruce Hogan spent the past 10 years in digital marketing and is the Co-Founder & CEO of SoftwarePundit, an internet business that helps SMBs adopt the best software for their needs. He’s used a host of CRO solutions and offered this advice:

Free A/B Testing Tools. There are a few downsides of free tools compared to for-purchase solutions. For example, paid tools give technical teams can launch A/B tests via code (vs. visual editor). It also has more advanced targeting capabilities to run experiments at specific user segments. Bruce Hogan

Faizan Fahim, a digital marketer at ServerGuy who oversees all the digital content of the site, shares his thoughts:

Free A/B Testing Tools. Free tools are amazing. But Functionality is limited. Features are limited. But sometimes we need more data to understand our visitors. In those cases, we prefer paid tools that can offer us insights into the forever changing pattern of customer visiting habits. Faizan Fahim

Trevor Ford, currently Head of Growth at Yotta Savings, explains the limits of free A/B testing and when migrating to paid ones makes sense:  

Free A/B Testing Tools. Paid tools tend to have more native connections to external and third-party platforms.  I.e. sending test data into a payment processing platform or into a CRM.  Trevor Ford

Main Takeaway: Great Free Tools Available But You Get What You Pay For 

Holly Winters of BrandCave encapsulated it best:

Free A/B Testing Tools. When it comes to product A/B testing tools, you almost always get what you pay for. Paid tools offer a more specific range of features than free tools do. Holly Winters

This doesn’t even factor in the level of additional services that are included in software packages that are sometimes overlooked, like customer support, branded CRO communities you can tap into, learning portals hosted by paid tools to help you become a better tester and CRO practitioner. 

Lastly, a paid A/B testing platform like Convert Experiences can increase your conversion rates and your bottom line. Experimentation tools like Convert Experiences give you a clear ROI on your plan investment. When used to its full potential, the tool can pay for itself. 

The best way to check out a tool is by exploring it. Get full access to Convert for 15-days for free. A/B test, check out all our integrations and see why so many optimizers chose Convert as an Optimizely alternative.

Can someone say, “Paid equals better than free?”

There is a reason medium size and enterprise companies invest in paid tools. They know it is a strategic investment that will give them a return. 

The Bottom Line…

Holly Winters said it best, “You get what you pay for.” While free, freemium, and open-source tools have their place, they can’t 100% reach the robust functionality of a paid tool. 

So, if you are a business new to the experimentation world, and have small numbers of visitors coming to your site, free A/B testing tools are fine. But as you grow, you should consider using a tool like Convert Experiences that gives you a seamless experimentation platform that can support your optimization efforts and push for growth. 

What is your experience with free A/B testing tools versus paid?

Get a Taste of One of the Most Privacy Aware A/B Testing Tools Out There
Get a Taste of One of the Most Privacy Aware A/B Testing Tools Out There
Originally published January 13, 2021 - Updated January 28, 2021

Start 15-Day Free Trial Of Convert Experiences.

Quick form submission
Quick submission.
No credit card needed
No credit card needed.
You can always change your preferences later.
Check Icon
You're Almost Done.
Managing Marketing Team
Managing Tech Team
Hypothesizing Experiments
Coding and QA of Experiments
Convert is committed to protecting your privacy.

You've successfully submitted your details to start your Convert Experiences 15-day free trial.

We are setting up your account for you. You'll receive a welcome message to your registered email with your login access. Please explore the app on desktop for an optimal experience.