What are tested users?

Tested users are the number of site visitors participating in A/B tests you have running on the Convert Experiences platform. Let’s break down how we count users. The people who come to your site are termed as your site visitors . Not all site visitors though are Convert app tested users . Users are specifically the people who are bucketed into tests that you have running on your site. In counting users, we do one key thing. We disregard overlaps . So if there are 100,000 site visitors who see Test #1, and another 100,000 visitors who see Test #2… the total app tested users will be: 200,000 people if the two sample sizes (of 100,000 each) have no overlaps. That means no one person has seen both tests. Less than 200,000 if the two sample sizes (of 100,000 each) have overlaps. If both sample sizes have 50,000 people who are unique to one test and 50,000 people who have seen both tests, then the tested users count will be 150,000. Tested users count is thus the same as (or less than) your website’s total unique site visitors, depending on how you choose to bucket your visitors into tests. This method of counting unique users replaces the older (pre April 2020) counting of unique tested visitors which took overlaps into consideration and gave a net tested visitors count based on all the visitors to each test.

Does Convert Experiences apply sampling?

No. Convert Experiences reports are based on unsampled data. This means our stats engines consider all the data collected by your set-up to call winners, thereby bypassing the issue introduced by most digital analytics platforms.

Does Convert Experiences allow server side testing?

Convert Experiences started out as a client-side A/B testing tool . But over the years we have received multiple requests from users who would like to include server-side testing to their optimization mix. As a result we do support server-side A/B testing through the manual creation of a Convert custom cookie which can determine Experiment ID/Variation ID, and programmatically assign it to website visitors.

How well does Convert Experiences work with Single Page Applications (SPA)?

In the standard format Convert Experiences takes note of the URLs website visitors navigate to trigger experiments (and goals). But with Single Page Applications (SPAs), this is not the case. Thus SPAs have to be handled differently. Our tool is capable of doing that. We use custom code to start polling for both when an experiment should be triggered on a page, and when a goal should be fired. We have detailed documentation in our support repository to walk users through this process for various SPAs like React/Redux and Angular.

How can you track revenue/transactions for eCommerce stores using Convert Experiences?

Convert Experiences has robust revenue tracking and it is privacy friendly as well. If you have the latest version of Google Analytics ecommerce tracking running on your store and the Convert tracking code is duly added after the tag, then we automatically connect revenue tracking to your experiments. You can also set up a revenue goal manually, count multiple transactions per customer, and track revenue attributed to actions that are not accompanied by URL changes.

Will my page load speed be impacted by the installation of Convert Experiences tracking code?

Convert Experiences is one of the fastest loading A/B testing tools in the optimization space. Through the use of our patent-pending SmartInsert™ Technology, we can offer you the advantage of fast loading time, and no flashing. Learn more here .

Do I need to ask consent from site visitors before testing with Convert Experiences?

No As long as you use Convert Experiences in its default setting, you do not need to ask consent from your website visitors and browsers. This is because we don’t store any personal data in the default, and use only first party cookies. If however you need to target audiences by city, or enable cross domain linking, you will see in-app GDPR warnings that are designed to let you test to your fullest potential while staying on the safe side of privacy.

What are must-have features for an A/B testing tool?

An A/B testing tool is obviously a key part of any conversion rate optimization drive. In recent times though there has been a definite shift in the way leaders, businesses and experts perceive optimization & experimentation. The discipline of CRO is no longer restricted to only conversion rates. The sentiment is swaying towards: Customer Experience Optimization (CXO). Learning from smaller experiment wins in the short term to benefit in the long haul Leveraging testing to mitigate risks in decisions across web design, revenue generation, product development, and Ops. “Recruiting” experimentation champions not just in the C-suite but across the entire organization, and doing this in an organized way. These shifts prove that even though experimentation & A/B testing are a rigorous science, the success of experiments largely rely on people and the extent to which a culture of experimentation is inculcated in them. Do they obsess over a few lifts in the present or do they get the value of being truly experiment driven for robust decision-making and a more mature, sustainable business model? How this is achieved is going to be unique to an organization and not defined by “tools”. Hence A/B testing tools should first and foremost provide flexibility, reliability, and support. These vendors too should have a culture of putting people (users) first over constant, forced price upgrades, the bureaucracy of gated pricing, and the tendency to tier important features like APl access, integrations, QA by exorbitantly priced anual plans. Want to learn more about the culture at Convert? Come meet our team.

Trina Moitra, Author at A/B Testing Software

Trina Moitra

Trina Moitra is the head of marketing at Convert. She loves setting the voice and vision behind Convert's brand messaging, cementing the market vision for the company. In her spare time, she can be seen taking personality quizzes to understand herself better.

Topic:

Author:

A/B Testing 04 Oct, 2023

Experimentation is Human Nature™. Use This Realization to Fuel Your First Test.

By Simon Girardin, John Ostrowski and Trina Moitra

Curiosity drives us. Picture a child who dismantles a toy, not to break it but to understand how it works. That’s us. We don’t grow out of that. We all have an innate desire to explore, to understand, and to improve. Yet, for many, the word “experimentation” feels distant,...

Conscious Business 14 Aug, 2023

Entire Businesses Can Be Run by AI Without Human Intervention: Why We Need to Talk About This (+ Convert’s Stance)

By Trina Moitra and Uwemedimo Usa

To kickstart this piece, let’s establish a fact: We aren’t AI usage or development experts. And we do not wish to masquerade as such. Instead, we’re observers, just like you, trying to make sense of an exponential advancement in technology. There’s been a ton (conservative estimate) of hype around...

A/B Testing 10 Jul, 2023

Encourage Innovation & Secure Buy-In with a Balanced Portfolio of Experiments

By Trina Moitra

Buckle up, and hold onto your curiosity hats, folks, because we’re about to plunge into the world of A/B testing portfolios like never before! We’ll start by breaking down multiple concepts and clearing up a few myths and misconceptions, and then we’ll dive into the meat (or tofu for...

Optimization 25 Apr, 2023

The Impact of AI on Experimentation: How Are Real Optimizers Using Artificial Intelligence to Test and Learn?

By Trina Moitra and Uwemedimo Usa

Artificial intelligence enabled apps did not invade the scene in 2022. The casual mention of AI has been around for decades. From Shakey the Robot in the late 1960s to the “Expert System” in the 1970s, and of course, various virtual assistants (Siri and Alexa) and chatbots, we’ve always...

A/B Testing 31 Mar, 2023

The Lifecycle of a Prank. We “Optimized” April Fool’s Day. And It Worked!

By Trina Moitra

The grown up, virtual equivalent of “Boo! April Fools…” goes down on LinkedIn, pretty much every year. In 2023 we saw Daniel Foley Carter appoint himself as the Global Head of SEO for OpenAI. And the inimitable Frenchman Louis Grenier flippantly accepted the position of Chief Growth Hacker for...

A/B Testing 13 Feb, 2023

Disruptive Ideas for a Disruptive Year Winning in 2023 with Innovation & A-B Testing

Disruptive Ideas for a Disruptive Year: Winning in 2025 with Innovation & A/B Testing

By Trina Moitra and Uwemedimo Usa

2025 does not look like the year the chaos is going to stabilize into the ultimate post-pandemic “normal”. Disruptions will continue to abound. With plunging savings, rising debt, and the overall economic pessimism that’s coloring predictions, consumer behavior will change in significant ways. In the past, organizations could afford...

The Convert Experiences blog has a great mix of current news, information, and useful guides

The topics covered are varied, including transparency & privacy, technical writeups, best practices, and industry roundups, all while staying relevant to marketers and optimizers of all skill levels.

Articles are backed up by practical, real-world examples and vetted by industry experts, so new pieces are always relevant with the most up-to-date information. It is one of my go-to sources for CRO focused content!

Matt Beischel

Founder & CEO, CorvusCRO

Your Blog Section is a great resource for both brands and marketers

I love the content you publish at Convert. I appreciate the time you take to produce such insightful posts on various marketing-related topics.

The majority of posts include real examples from brands and marketing campaigns, which is a great value addition for your readers. I personally enjoy many of your expert roundup posts, case studies, and the latest marketing tactics you cover.

Shane Barker

Digital Marketing Consultant and strategist

Trina Moitra

Experimentation is Human Nature™. Use This Realization to Fuel Your First Test.

Entire Businesses Can Be Run by AI Without Human Intervention: Why We Need to Talk About This (+ Convert’s Stance)

Encourage Innovation & Secure Buy-In with a Balanced Portfolio of Experiments

The Impact of AI on Experimentation: How Are Real Optimizers Using Artificial Intelligence to Test and Learn?

The Lifecycle of a Prank. We “Optimized” April Fool’s Day. And It Worked!

Disruptive Ideas for a Disruptive Year: Winning in 2025 with Innovation & A/B Testing

Product

For CRO Agencies

Resources

Company

Important. Please Read.

Convert Blog: Learn A/B Testing, Optimization & Marketing

Trina Moitra

Blog Posts

Experimentation is Human Nature™. Use This Realization to Fuel Your First Test.

Entire Businesses Can Be Run by AI Without Human Intervention: Why We Need to Talk About This (+ Convert’s Stance)

Encourage Innovation & Secure Buy-In with a Balanced Portfolio of Experiments

The Impact of AI on Experimentation: How Are Real Optimizers Using Artificial Intelligence to Test and Learn?

The Lifecycle of a Prank. We “Optimized” April Fool’s Day. And It Worked!

Disruptive Ideas for a Disruptive Year: Winning in 2025 with Innovation & A/B Testing

Important. Please Read.