The simple promotional bar that led to a 20% increase in an online store's revenue.

A luxury handmade glass brand launched promotional sales campaigns offering its buyers free shipping. While the conversion rate increased during such promotional drives, the team wanted to know if it increased enough to boost the overall store profitability (given the additional shipping costs that the store had to bear). The brand engaged San Francisco eCommerce growth agency Brave One to find out the truth behind the positive numbers.

For an eCommerce store, it’s not just important to make more sales. It’s also important that the increased sales translate to more revenue — which may not always be the case. Brave One quickly sprung into action to see if and how the free shipping offer impacted the store’s bottom line.

Hypothesis:

For the brand, the real question (or the real test) was to find out if the free shipping offer raised the sales enough to offset shipping costs while still increasing revenue.

Brave One wrote a hypothesis focused solely on the free shipping offer’s revenue potential.

The client had been using a Free Shipping offer to drive sales and promotions. They knew the conversion rate increased during these times, but they didn’t know if it increased enough to increase profitability due to the associated increase in shipping costs.

Lukas B. Snelling

Brave One Agency

Brave One hypothesized:

Free shipping increases conversion rates even outside of promotional periods, and enough so that the increased volume will make up for the difference in shipping costs to the client.

Lukas B. Snelling

Methodology

Spread:   50/50 a control and a challenger.	Visitors:   -25,000
Confidence Score:   99.26%	Time:   15 days

Treatment:

Based on its hypothesis, Brave One added a promotional bar to the store’s website offering free shipping for a limited time (on orders over $250). The idea was to immediately get the visitor’s attention to the free shipping offer.

The Challenger:

The promo bar from the challenger:

The Control

Brave One tested the challenger against the original version over a period of 15 days.

Each version saw about 25,000 visitors. The large sample size and the long test duration ensured that the results of the experiment were consistent and statistically significant (and not random false positives).

The test included traffic from all sources (such as paid, social media, referral, organic and other channels) and comprised of high-income individuals.

Brave One used Google Analytics and Mixpanel for getting the website analytics data and Convert to design and run the experiment.

We use Convert.com because it offers deep integrations into the tools we use like Mixpanel and Google Analytics. Taken together, this allows to have a robust understanding of user behavior and demonstrate clear results to our clients.

Lukas B. Snelling

Results:

Brave One saw a whopping 20% increase in the revenue for the brand’s online store. The challenger beat the control and raked in an additional revenue of ~ $16,000.

We believed the free shipping offer would win – but the question was by how much – and could the additional increase in conversions increase overall profitability.

Lukas B. Snelling

Brave One always believed that the free shipping offer would win, but with the test, Brave One confirmed that the additional sales that the free shipping offer brought easily made up for the shipping costs and drove a spike in profitability.

We learned that free shipping increased revenue and transactions enough to overcome the impact on the margins of increased shipping costs.

Lukas B. Snelling

The test reinforced the idea that users don’t like to pay for shipping and prefer free shipping. The successful experiment showed that the brand could launch buyer-friendly shipping promos while actually making profits (despite the shipping costs).

The Big Takeaways: From the Obvious to the Revealing

1. Consider offering free shipping. Free shipping is a huge incentive for online buyers. If you can offer it (and if your competitors offer it), consider offering it. Using Convert to determine the true impact on the profitability of such an offer is a great way to make a business case for this pro-consumer policy.
2. Don’t assume; test. In the case of this brand, the free shipping offer brought additional revenue as well as more sales. But this may not be the same for you. A promotion that drives more sales won’t necessarily bring greater profit. The ultimate goal of any promotional exercise should be to balance the spend with the profit.
3. Understand your customers. Think about what gets your customers to buy. Coming up with offers like free shipping is only possible when you understand your customers’ buying motivations.
4. Testing isn’t just for optimization. It’s also a tool for accelerated business growth. This is perhaps the biggest takeaway from Brave One’s efforts. Testing is a versatile tool. It can definitely increase conversion numbers. But it can also help take data-driven, confident decisions in sales and marketing.