Are you one of those people using AI for analysis and statistical calculations? I know why you might be doing that. Convenience, right? Reducing friction. But let's consider the story of the data. That hard-earned data you spent time and effort ensuring robustness and accuracy. And then you throw it into a language model and…
Read the full article→
We solemnly promise that each bit of content you read here
is thoughtfully curated or crafted by a human, for a human. We spend hundreds of hours every month speaking to experts with decades of experience. Our blogs are more a journalistic report on the state of experimentation than a single perspective article. Enjoy!
Forecasting the effect your experiments will have on future revenue is a bit of a controversial topic in the CRO world. The reason for this comes down to the fact that we don't really know for sure if a 10% lift from an experiment will translate into a 10% increase...
Work the pyramid from the bottom up When you think of first running price tests, what comes to mind? Adjusting your prices up or down? Maybe testing discount levels? That’s what comes to mind for most of us, but it should be the last thing you test. Most subscription apps...
Buse Hizarci, Optimisation Manager at Vervaunt, is the latest expert to open up their playbook for our Testing Mind Map series. These interviews look past the tidy version of experimentation and into the day-to-day: how evidence fares against conviction, which test wins survive contact with production, and where judgment sits...
Feature flags switch functionality on or off at runtime, without a deploy. Rollouts widen an already-approved feature to a growing percentage of users. Feature testing compares variations against a control to find which performs better, using A/B/n testing underneath. All three run on feature flags. Only feature testing holds back...
Let’s set the stage. A CRO team runs an A/B/n test with one control and four variations, tracking three success metrics across each layout. One metric on Variant C clocks in with a p-value of 0.04. The team declares a winner, ships the change, and waits for the revenue to...
Craig Kistler, who leads experimentation at Signet Jewelers, is the latest expert to open up their playbook for our Testing Mind Map series. These interviews look past the tidy version of experimentation and into the day-to-day: how evidence fares against conviction, which test wins survive contact with production, and where...
