Loop Earplugs & Dash of CX: Personalisation Tested, Insights Gained

The Challenge

Loop Earplugs wanted to enhance their direct-to-consumer offering by introducing a personalized carrying case. The concept was simple: for an additional €6, customers could engrave the case with custom text, creating a unique product experience.

The primary objective was to encourage customers to buy directly from their website instead of through third-party retailers, like Amazon. The hypothesis was that the personalized case would increase customer satisfaction, differentiate their brand, and boost key metrics like Revenue Per Visitor (RPV) and Average Order Value (AOV).

Adding an extra feature raised a critical question: would the added step create friction in the checkout process, potentially disrupting the user journey?

The Experiment

The agency “Dash of CX” designed an A/B test to evaluate the impact of the personalised case option:

• Version A: Included the option to personalise the carrying case during checkout.

• Version B: Maintained the original checkout flow without personalisation.

Test Details:

• Metrics Measured: Revenue Per Visitor (RPV), Conversion Rate (CR), Add to Cart Rate, and Average Order Value (AOV)
• Audience: Mobile shoppers interested in purchasing earbuds
• Traffic Split: Evenly split, 50/50
• Duration: 28 days

The Results:

Version B, the original checkout flow without the personalised carrying case option, outperformed Version A across all key metrics. While the personalisation option generated interest, with 6.89% of users clicking on the new call-to-action, this engagement did not translate into conversions. Instead, the added step introduced friction that negatively impacted essential KPIs:

• A 3.04% drop in Conversion Rate
• A 2.96% drop in Revenue Per Visitor (RPV)
• A 5.68% reduction in Add to Cart conversions

Test Details:

• Timeframe: 28 days
• Traffic: Over 300,000 mobile users, evenly split between the two versions (50/50)

These results demonstrated that introducing additional decision-making steps at the checkout stage disrupted the user journey. The findings provided clear evidence that personalisation, when placed at the wrong point in the journey, can have unintended consequences.

The results were statistically significant at 99% confidence with no sample ratio mismatch (SRM), confirming the reliability of the findings.

Key Learnings:

Personalisation: Timing Is Everything
The test revealed that customers engaged with the personalisation option, as evidenced by a notable clickthrough rate on the new call-to-action. However, this interest didn’t translate into conversions, highlighting that timing and placement of such features are critical.

Introducing such features earlier in the user journey, on the Product Detail Page (PDP) or mini-cart, may allow for greater engagement without detracting from purchase intent.

The Value of Testing

This test reinforced Loop Earplugs’ approach to using A/B testing for updates, particularly for innovations like personalisation and add-ons. Chris Marsh from Dash of CX highlighted:

“It shows there’s a significant risk of harm to UX and revenue metrics if updates are launched without testing. The percentage of tests that are flat or can hurt KPIs is generally much higher than stakeholders would guess!”

The results demonstrated the necessity of testing to avoid unintended consequences and make data-driven decisions that protect both user experience and business performance. Another important gain from this test was that Loop learned they didn’t need to stay with the logistics warehouse offering engraving, giving them more strategic freedom.

The Role of Convert:

• Microsoft Clarity Integration: “The ability for Convert to automatically push data to Microsoft Clarity is a game-changer,” shared Chris Marsh from Dash of CX. “It enables us to easily review session and heatmap data and spot possible UX issues.”
• Segment Analysis: Convert’s segmentation tools were useful in understanding the nuances of user behaviour. As Chris explained, “Segment analysis is important to see how the overall pattern differed between traffic sources, devices, and new and returning users.”
• Responsive Support: “Convert support is always very quick to respond when we have technical questions, helping us test with speed” said Chris. The responsive assistance ensured the team could focus on delivering robust and timely results without unnecessary delays.

The Way Forward

This experiment has inspired Loop Earplugs to continue exploring personalisation and add-ons as part of their optimisation journey. Future tests could focus on introducing such features earlier in the funnel, where they are less likely to disrupt the purchasing process.

Dash of CX’s approach has not only informed Loop’s current strategy but also set the stage for further innovations, ensuring features are tested to confirm they align with user needs and expectations.

This test won GOLD in the guessthetest.com “A/B Test Case Study Awards”. Thank you to Deborah O’Malley and Dash of CX for sharing this test with us.