What Goes Into Consistent Optimization Success: The Anatomy of the A/B Testing Framework Used by CRO Experts

Justin Christianson
September 6, 2018 ·

Conversion optimization, if done correctly, is one of the most beneficial and impactful things you can do for your business.

When compounded – CRO, according to a recent study, returns on average $45 for every dollar spent and those that leveraged an agency saw 1 in 3 tests as improvements as compared to 1 in 6 doing it themselves.

We see this all the time with our clients, 20, 30, 40% or more increases in key metrics like conversion rates, revenue per visitor and micro commitments like cart views.

But to effectively optimize you need to be more strategic than just testing random elements in hopes that you will find a winner.

Today we are going to discuss what an efficient CRO methodology looks like so you can ensure you are getting the most out of your efforts.

Our philosophy at Conversion Fanatics is pretty basic as we don’t like to overcomplicate things. We don’t use fancy acronyms or crazy frameworks to get results for our clients.

Our process has been fine-tuned after years of experimentation and thousands of market tests and when boiled down is quite simple.

Think (Analyze) – Build (Design) – Run (Experiment)

Think (Analyze) - Build (Design) - Run (Experiment)

Leverage both qualitative and quantitative data

This will help you determine what the visitors are paying attention to, what they are ignoring, and where they are falling off in the process to the end goal you want to achieve.

We use this data as well as data collected from both winning and losing experiments to guide the process.

Don’t get discouraged by losing tests either as they can often teach us more about the visitors behavior than the winners.

The main goal we are looking to achieve is to measure through experimentation what holds the most weight on the page in the eyes of the visitor and remove any major friction points that could cause them to not take the desired action you want them to take.

Mix this in with some strategic testing to drive KPIs such as average order value and you have a winning formula for long-term exponential growth.

You never want to run an experiment without first establishing a hypothesis.

A hypothesis is basically answering the question “why?” What change do we want to make, what impact do we feel it will have as well as why we feel that is the case?

When you constantly ask yourself “why?”, the optimization process becomes more valuable.

As an example, say you want to test something as simple as a button color, instead of just knowing that a green button out performs an orange one, you would establish a hypothesis which would look something like this.

Changing the button color from orange to green will increase add to cart clicks because the color change adds more contrast to the main call to action to encourage the click.

Seems simple but we are changing our thinking from just a random test to truly trying to understand why the change had an impact.

I can look at a site and make a list of a hundred things that could be tested but when you are looking for impact and are asking the right questions that is where the better test ideas are created.

One thing to note though… many people feel that bigger changes will yield bigger results when in fact that isn’t always the case. Bigger just means it is going to take more time to develop and could actually not win. Instead we always recommend testing incremental elements to gauge the importance of them in the eyes of the visitors which will often inform bigger tests that can actually have a bigger impact.

What most people don’t understand with CRO is that it takes time so you have to be patient, it is a process, it’s hard work and you won’t always hit a home run so you need to learn from every test.

Create Hypotheses
Create Hypotheses

Follow the data, be strategic, expect to lose but don’t get discouraged.

Testing is more than just increasing your conversion rate, there are times when you increase conversion rate and actually make less money. So you need to measure other variables like revenue per visitor, average order value and micro conversions such as button clicks.

A great byproduct of effective optimization is you will have happier customers, you will get more repeat buyers and your customer acquisition costs may go down as your visitors convert at a higher rate and will often spend more money.

We took a look at what goes into an effective process for CRO, but what does the flipside look like?

As mentioned, CRO is a process that takes time but can lead to exponential growth when done right. Where people fall off, get discouraged and eventually count out CRO is when they go at it without a process in mind.

Tell me if this sounds familiar… You test a few elements, you win occasionally but are now finding it hard to have any meaningful impact, you quickly get frustrated and your testing efforts get pushed down the priority list?

It happens to even the best of companies, testing doesn’t have the immediate impact and they quit or don’t focus on it.

You need to stick it out and follow a process instead of randomly testing things to see what sticks. As an example a recent client of ours… we ran the first 11 tests without a meaningful impact and on the 12th test we increased their sales conversions 52%.

We didn’t give up, we learned from every losing test, we figured out what the visitors wanted and we ultimately came out on top.

So bottom line, don’t randomly test.

Always ask yourself “why” and learn from every test you put out there.

If you do this you will open up much bigger opportunities for growth.

If you would like to find out where you are falling short by getting a fresh set of eyes on your marketing and analytics I encourage you to go to the link below to request your free, no obligation in-depth conversion analysis. We dive deep on your analytics and site and give you an action plan to be more effective optimizers.

Claim Your Free Site Evaluation @ http://conversionfanatics.com >>

Originally published September 06, 2018 - Updated July 17, 2024
Mobile reading? Scan this QR code and take this blog with you, wherever you go.
Justin Christianson
Justin Christianson Justin Christianson is the Founder of Conversion Fanatics

Start Your 15-Day Free Trial Right Now.
No Credit Card Required

You can always change your preferences later.
You're Almost Done.
I manage a marketing team
I manage a tech team
I research and/or hypothesize experiments
I code & QA experiments
Convert is committed to protecting your privacy.

Important. Please Read.

  • Check your inbox for the password to Convert’s trial account.
  • Log in using the link provided in that email.

This sign up flow is built for maximum security. You’re worth it!