Design A/B tests with clear statistical guardrails before the test begins and stick to them during analysis. Recommended practices: ● Pre-calculate sample size and test duration so you don’t stop tests early ● Define one primary metric and treat others as guardrails ● Apply Bonferroni or Sidak correction for multiple variants ● Avoid post-test segmentation…
Read the full article
We solemnly promise that each bit of content you read here
is thoughtfully curated or crafted by a human, for a human. We spend hundreds of hours every month speaking to experts with decades of experience. Our blogs are more a journalistic report on the state of experimentation than a single perspective article. Enjoy!
How to test, optimize it, and use AI for better performance One of my favourite parts of a Product Detail Page (PDP) is the product image carousel. Not because it’s fancy, but because it’s a small space where you can pack in a huge amount of decision-making support. And...
In 2026, easy access to experimentation tools has lowered the barrier to running A/B tests. Execution is now really cheap, and AI can now generate ideas, build variants, write code, and summarize results in seconds. The competitive edge comes from rigor, transparency, disciplined decision-making, and building a process your...
The A/B testing and experimentation space looks very different today than it did 5–10 years ago. Private equity firms are consolidating independent tools, tech giants are acquiring experimentation infrastructure outright, and AI is transforming what it means to “run an experiment.” This article explores the major acquisitions, mergers, and...
Interview with Pieter Boonstra There’s the experimentation everyone talks about. And then there’s how it actually happens. We’re hunting for signals in the noise to bring you conversations with people who live in the data. The ones who obsess over test design and know how to secure buy-in even...
Part of the Small Models series: This article explores one way to interact with the Convert MCP server. It’s an advanced use case for those who want to experiment with small, cost-efficient, and secure models and alternative tooling. In my previous article, I showed how to run Convert Experiences...
Part of the Small Models series: This article explores one way to interact with the Convert MCP server. It’s an advanced use case for those who want to experiment with small, cost-efficient, and secure models and alternative tooling. Let’s do something cool with Model Context Protocol (MCP) today. If...
