In its first three months with a global leisure and tourism business, Another Web is Possible (AWIP) launched 16 A/B tests and found four clear winners. The best performing experiment raised revenue per visitor by 12%.
AWIP’s Jon Crowder calls the Convert MCP Server “an essential part” of how he builds and monitors experiments for such unprecedented agility.
He has plugged AWIP’s own CRO technology into Convert Experiences through it and says he now builds, QAs and launches in a fraction of the time that it used to take by hand.
Results
| Measure | Previous program | First three months with AWIP |
| Tests | 12 in 12 months | 16 |
| Win rate | 33.33% | 25% |
| Winners | About 4 in 12 months | 4 |
| Biggest win | Unknown | +12% revenue per visitor |
[**Source for the numbers: AWIP’s own case study. The previous program’s winner count is our arithmetic from 12 tests at a 33.33% win rate. AWIP also reports that the results of its first experiment paid back more than the client will pay for the program for the entire year]
Where the client started
The client’s previous agency ran roughly one experiment a month. When every test takes that much effort to launch, tests become a precious resource and everyone tries to filter the backlog. They do this without evidence searching desperately for the one idea that will “definitely work”.
Meanwhile, website changes go around the testing program and are launched on instinct, bypassing the precise measurement that experimentation gives you. It’s a new offer here, a new banner there. This constrains what experimentation can do to transform a business and the team who have the explicit target to grow online sales mostly ends up campaigning-for and delivering changes that do not, or can not, be shown to do that.
What AWIP changed
AWIP identified the source of those tensions to be one thing: how difficult it was to get a test out the door. They came in as the experimentation expert alongside a digital product partner already providing research and design for the client.
- A documented hypothesis for every idea. Each idea got a real, falsifiable hypothesis and was published in a portal open to the whole business. Radical transparency showing that they have nothing to hide.
- Ideas scoped for the whole estate. Ideas that were fed in from specific areas or pages were broadened so they could be tested across the entire estate. Each experiment developed had a rollout plan for what would be done in the event that it won.
- An experiment request form. AWIP shared this across the business. Ideas now flow through the program because it is the fastest route to a change, and their stakeholders are being actively engaged from all areas of the business on experimentation as a clear path to sustainable growth.
How the outcomes map to Convert Experiences and its MCP server
Jon Crowder builds and monitors experiments through the Convert MCP Server, with AWIP’s own CRO technology plugged into Convert Experiences through it.
| Outcome | In Convert Experiences | Via the Convert MCP Server |
| 16 tests launched in three months | A/B experiences can be built from custom JavaScript and CSS, each with its goals, audiences and locations. | An AI assistant can create draft A/B and Split URL tests, update variation code, and create or update goals, audiences and locations. |
| Bigger tests than the client expected, inside Convert | Variation code is applied in the visitor’s browser by the Convert tracking script. In the client’s own words: “I didn’t fully appreciate the scale of the test you could do within Convert itself versus needing dev effort.” | The server can update that variation code. It validates every write before it reaches the account, and it allows for much more ambitious experiments and prototypes at much lower cost. |
| Four clear winners, with +12% revenue per visitor on the biggest | Revenue tracking, reports under Frequentist, Bayesian or Sequential statistics, and a sample ratio mismatch check that is switched on per project. | Using MCP, we get the power of Convert’s stats engine and per-experiment metrics and can connect it to our wider stack really efficiently for behavioural post-test analysis. |
Who stays in charge
The winning hypotheses came from an in depth understanding of the user and their goals. Multiple pieces of research were conducted, including a detailed journey analysis. AWIP found that visitors who had not yet purchased were ending up in information and support pages that the business considered to be designed for customers who had already purchased.
That insight, among others, led to four conclusive, winning experiments.
The Convert MCP Server defaults to read-only reporting mode, and write access has to be turned on by the user.
It can create tests as drafts, and it validates each write before it reaches the account. Activating or pausing an experience through it requires explicit confirmation. Where an account uses Convert’s Request Approval System (beta, in Pro and Enterprise plans), the server keeps that requirement in place and never approves a request automatically.
A technically lower win rate, but many more wins and a very happy customer.
AWIP’s win rate is 25%. The previous agency’s was 33.33%. The client is delighted anyway because at one test a month, an inconclusive test is a disaster yet at AWIP’s pace it is, in their words, “just another day.”
Losing tests always teach us something too. For example, one showed that some tabs AWIP had initially treated as a fairly standard part of the navigation were behaving like merchandising functionality and that finding drove a run of follow-up tests.
In their words
The Convert MCP is an essential part of my experiment building and monitoring workflow. What an MCP does is expose the different features and tools of a service in a way that can be easily used by AI tools. Think of it a bit like an API but much more flexible.
With an MCP as fully featured and well built as the Convert MCP, I’ve been able to build proprietary CRO-focused expert technology for Another Web is Possible and plug it straight into the experimentation powerhouse that is Convert.
This lets me build and deliver effective and advanced experiments in a world-class tool in a fraction of the time it used to take writing the code by hand or using a WYSIWYG editor.
Jon Crowder, Another Web is Possible
It’s very good to see what a CRO process that is robust, streamlined, efficient, and really clean and neat and clear in its approach can do … it’s getting a lot of positive feedback from SLT and C-suite members as well.
The client, quoted in AWIP’s case study.
Related case studies:
nsightWhale and a destination travel company: a form cut from eight visible fields to three, and 26% more conversions on the deal page.
Another Web is Possible (AWIP) is a UK agency and Convert Certified Agency Partner focused on leaving behind dark patterns and extractive CRO practices. Jon Crowder - founder and chief optimiser - believes in working with evergreen levers that lead to sustainable growth through a mix of digital strategy, psychology, research, and experimentation: anotherwebispossible.co.uk
Convert Experiences is the preferred A/B testing & personalization tool for lean growth & marketing teams, CRO agencies, and brands in tightly-regulated industries like insurance. GDPR-compliant, CCPA-ready, ISO 27001 and SOC-2 certified, it offers easy consent management, only first-party cookies, in-app warnings to maintain privacy, anonymization, data segregation and more at publicly-listed, affordable prices.