Solving the Fragility of Redirect Tests with AI-Driven Regex Automation
I’m going to be honest: I’m not a huge fan of redirect tests. It’s tough maintaining data integrity in those things. I’m specifically talking about client-side redirect tests, by the way. Server-side redirect tests are fine.
Redirect tests are when you have a page (e.g. /my-original-page) and want some of your traffic to be redirected to a different URL (e.g. /my-new-page-version). They are also referred to as “split URL tests”. I’ll be using the two terms interchangeably for the rest of the article.
Whether you’re doing this client-side or server-side, the mechanics are usually the same: it involves writing some sort of “regular expression” as a rule to match a URL a user is trying to reach, and instead redirect those users to the new URL.
The problem is, regular expressions are brittle. A small typo, an unwanted URL in scope, or a missing edge case, and you have a very serious defect on your hands.
This is why I favour a very strict process for creating and QAing these kinds of tests. And a process such as this is perfect for automation.
This is why I’ve created an n8n workflow to handle these builds. There’s a link below to download the n8n workflow.
In the rest of the article, I’ll take you for a tour of how it works, in case you want to customise it for your own use (and you really should).
How the Workflow Works
Step 1: Collect the Required Information
The workflow starts with a form that captures three things:
- The target URL – the page you want to redirect
- The change request – a plain English description of the change
- The sitemap URL
A form is the best way to guarantee you’re getting the right information every time. You could also trigger this from a chat or other tool, but the form keeps things clean and enforces the information in the simplest way.
The next node just sets all the variables in a convenient way for subsequent steps.
By the way, you need a sitemap for this, or a list of all URLs on your site. If you don’t have a sitemap, either fix that today, or customise this step to work with a list of URLs to validate.
Step 2: Fetch and Prepare the URLs
The next set of nodes fetches all URLs from your sitemap and formats them into a list. That list then feeds into the next step.
Step 3: Determine Scope (AI Step 1)
This is the first place where AI comes in:
An agent looks at the full list of URLs, your target URL, and your change request, and determines which URLs should be in scope for the redirect and which shouldn’t.
The prompt works based on patterns, as you’ll see if you open the Agent node:
Prompt:
I am creating a Regex.
Output comma-separated URLs from the URL list that are in scope to be be changed.
%INPUT%
URL_LIST: https://mysite.com/,https://mysite.com/page,https://mysite.com/page2,https://mysite.com/page/sub-page
TARGET_URL: https://mysite.com/page
CHANGE_REQUEST: Change /page to /page-alt
%OUTPUT%
https://mysite.com/page,https://mysite.com/page/sub-page
%INPUT%
URL_LIST: {{ $json.urls }}
TARGET_URL: {{ $('Edit Fields').item.json.url }}
CHANGE_REQUEST: {{ $('Edit Fields').item.json.change_request }}
%OUTPUT%
I like this structure for my prompt. A clear task, and a few examples for the LLM to pattern match against (this is called “few-shot” prompting, by the way). We’re going to leave the decision on which URLs to include and which to exclude up to the LLM. I’m using a small model here, and this seems to be doing a decent job in my tests.
It’s not perfect, but it’s better than doing it manually across hundreds of URLs. You can optionally include a human-in-the-loop check here if you want.
Step 4: Write the Regex (AI Step 2)
This is the second and main AI step. The agent takes the scoped URL list and writes a regular expression. The prompt is the same structure: a task instruction, some pattern examples, and then the actual information to use.
Prompt:
Regex to change url to Change Request. Preserving all paramaters. Always use the tool to validate the regex, fix as neccesary. No lookbehind. Ensure regex matches with / at the end.
{{ $('Merge').first()?.json?.failures ? "Previous attempt failed. Fix these issues:\n" + JSON.stringify($('Merge').first().json.failures, null, 2) : "" }}
%INPUT%
URL: https://mysite.com/?product=myprod
Change Request: product=myprod → product=myprod-alt
%OUTPUT%
{"match": "(^|[?&])product=myprod(?=&|#|$)", "replace": "$1product=myprod-alt"}
%INPUT%
URL: https://shop.example.com/cart?utm_source=email&ref=blog
Change Request: remove the ref query parameter entirely
%OUTPUT%
{"match": "[?&]ref=[^&#]*(?=&|#|$)", "replace": ""}
%INPUT%
URL: https://blog.example.com/posts/summer-sale-2024
Change Request: replace slug summer-sale-2024 with summer-sale-2025
%OUTPUT%
{"match": "\/summer-sale-2024(?=\/|$|\?|#)", "replace": "\/summer-sale-2025"}
%INPUT%
URL: https://app.example.com/users/88421/settings
Change Request: swap numeric user id 88421 for 10001
%OUTPUT%
{"match": "\/users\/88421(?=\/|$|\?|#)", "replace": "\/users\/10001"}
%INPUT%
URL: {{ $('Edit Fields').item.json.url }}
Change Request: {{ $('Edit Fields').item.json.change_request }}
%OUTPUT%
Small models work fine here, too.
Step 5: QA the Regex
This is where the script gets validated.
A script runs through every single URL from the sitemap, applies the regex, and checks:
- Does it match URLs it’s supposed to match?
- Does it leave alone URLs it’s supposed to leave alone?
If there’s an error, it formats that error, sends it back, and the workflow loops back to rewrite the regex.
The error adds additional context for the Agent to do its update. This loop continues until it passes QA.
Step 6: Create the Split URL Test
Once the regex is validated, the workflow creates the experiment in your testing tool (in this case, Convert). It sets the experiment name, creates the variation, and enables the regex. Again, small models run very quickly. This is the part you might want to adapt if you’re not a Convert customer (this is why I’ve marked this red).
Results of my tests
The whole process runs very fast. It’s also very cheap. The number of tokens used is minimal, and the output is solid. I used an example where my change request lacked clarity in the URL scope to include, and the workflow still picked up URLs to include and ensured a robust regular expression was written, excluding out-of-scope URLs.
Why All of This Matters
This workflow eliminates a lot of the user error that makes redirect tests so fragile in the first place. It’s not going to replace proper QA, but it should reduce some effort. In fact, if you have a QA, I’d recommend appending a step to write the QA requirements and documentation (I removed this from the final workflow to keep things simple).
My intention here is not to give you a final n8n workflow you can pick up and apply; rather, it is to give you some ideas for rolling out similar, reliable processes that can be used by the whole team with limited specialised knowledge required.
Editor’s note: This guide is part of a broader series on building practical AI systems. Since you’re reading about A/B test automation, you’ll likely want our related guides on exploring Convert MCP with small models using Claude Code and Qwen 3 and turning Convert MCP into a secure n8n workflow with MCPO. If you’re just getting started more broadly, we’d also recommend our guides on getting started with AI automation in n8n, building your first AI agent, building RAG workflows with n8n and Qdrant, connecting chat interfaces to other tools using MCP, and setting up MCP servers in n8n.
Written By
Iqbal Ali
Edited By
Carmen Apostu
