Testing Mind Map Series: How to Think Like a CRO Pro (Part 69)

Brian Poe
By
January 13, 2025 ·

Interview with Brian Poe

Every couple of weeks, we get up close and personal with some of the brightest minds in the CRO and experimentation community.

We’re on a mission to discover what lies behind their success. Get real answers to your toughest questions. Share hidden gems and unique insights you won’t find in the books. Condense years of real-world experience into actionable tactics and strategies.

This week, we’re chatting with Brian Poe, Chief Optimization Officer at Mammoth Insights and Author & Host at EXO Field Notes Newsletter and Experimentation Roundtable.

Brian, tell us about yourself. What inspired you to get into testing & optimization?

I’ve built my career in digital analytics and optimization over the past 14 years here in Minneapolis, Minnesota. My journey began at Target Corporation, where I partnered with engineers and consultants to develop technical and solution-focused documentation for Omniture SiteCatalyst, honing my skills in website tagging and tracking. From there, I eventually moved to Best Buy, where I had the privilege of leading their experimentation program and managing a talented team of CRO and digital analytics analysts. Over the years, I’ve worked with a number of digital agencies, where I’ve managed and consulted on CRO and experimentation programs for organizations such as Verizon, Ulta Beauty, and CuriosityStream, Marriott, among many others.

What inspired me most to pursue a career in testing and optimization is the intersection of creativity and data. My first university degree is in graphic design, so I’ve always gravitated toward creative work, even before stumbling into this industry. I’ve always been fascinated by how experimentation can uncover unexpected insights and empower businesses to make smarter decisions. Today, I value the opportunity to lead programs, mentor individuals in their careers, and channel my entrepreneurial spirit into developing innovative solutions that drive growth and transformation.

How many years have you been testing for?

I’ve been running online experiments since 2012, when I was working at a small agency and incubator in Minneapolis. Back then, I was testing acquisition messaging and product features for startups, which was my first taste of how powerful experimentation can be. Not long after, I took on a role at Capella University, where I led their web analytics work and eventually their testing and optimization program.

The real turning point for me came at Best Buy, where I led a team dedicated to experimentation. I loved collaborating with product, media, and marketing teams to tackle unique and complex testing challenges. Building organizational processes and evangelizing the value of experimentation across a matrixed organization was especially rewarding. Since then, I’ve had the chance to lead programs for many companies (mentioned above), and I’d say I’m a veteran in this space now—but I’m still learning and evolving every day.

What’s the one resource you recommend to aspiring testers & optimizers?

My top advice isn’t a book, course, or tool—it’s learning how to build and navigate relationships within your business. Relationships are often overlooked in this industry, but they’re the most underrated skill set you can develop. Hard skills like tools and analytics can be taught (and many will eventually be automated), but the ability to connect with others, collaborate, and influence will always be valuable. It’s a timeless skill that won’t be disrupted by technology, and it will serve you well in both your career and your life.

That said, a book I’d recommend exploring is Connect: Building Exceptional Relationships with Family, Friends, and Colleagues. While I haven’t read it yet, it comes highly recommended, and I’d love to hear your thoughts if you dive into it. If I get a chance to read it myself, let’s compare notes—it sounds like a great resource for anyone looking to strengthen their relationship-building skills!

Answer in 5 words or less: What is the discipline of optimization to you?

Data-informed decisions, empowered relationships.

What are the top 3 things people MUST understand before they start optimizing?

Most people approach this question with technical answers like statistics, psychology, or UX best practices. While those are important—they’re the foundation of any solid program—they won’t make a difference without buy-in from leadership, support for time and resources, and a shared commitment to making data-driven business decisions.

At its core, optimization is about moving the business forward in a meaningful way. Building relationships is the real equalizer, not just how much data you have or how technically skilled you are. As an optimizer, your first focus should be on people: earning trust, fostering collaboration, and aligning teams around shared goals.

Once you’ve established that trust and buy-in, the tools and processes will naturally follow. If you’re working in isolation or feel like no one is listening to your recommendations, the program is set up to struggle from the start. Prioritize people, build a strong foundation of support, and then leverage your technical skills to drive impactful change.

How do you treat qualitative & quantitative data to minimize bias?

I believe in the saying, “Where there’s smoke, there’s fire.” While qualitative and quantitative methods answer different types of questions, their shared goal is to uncover evidence for change. If qualitative research reveals that users are frustrated with your product, and quantitative data supports that finding, it’s a signal worth paying attention to—and potentially acting on.

That said, research is about balancing time and resources. Like experimentation, it’s a tool to mitigate and prioritize risks, not a one-size-fits-all requirement for every organization. Use research to guide decisions, but don’t let the pursuit of data lead to analysis paralysis. Sometimes, you need to move forward with the information you have and refine as you go. The key is to focus on making informed, actionable decisions without overcomplicating the process.

How (to you) is experimentation different from CRO?

This has been a debated topic over the past few years, and while I have my own thoughts (and a few pet peeves) about the terminology, I think the distinction is less important than some might believe. If you’re on the agency side, your buyers don’t usually care about the nuances—they care about results and how you deliver them.

While there are similarities and differences between experimentation and CRO, the market will ultimately define these terms, not us as practitioners. What matters most is focusing on driving meaningful outcomes and creating value, regardless of the label you use.

Talk to us about some of the unique experiments you’ve run over the years?

Let me share a recent example and a resource I think your audience will find valuable.

I’ve been working with a travel company on an exciting experiment that combined media testing, landing page design, and experimentation. It’s rare—especially on the agency side—to collaborate with a media team on a holistic optimization strategy, which made this project interesting.

The start of the experiment was a completely new landing page design that we believed could outperform the existing one. From there, we layered a series of bold thematic tests, focusing on message strategies that were designed to either succeed significantly or fail outright—no incremental changes here. We also faced a tight timeline, so every test needed to be planned and ready to launch the moment the previous one ended. For the final phase, we had to prepare two scenarios (A and B options) in advance, which required extra effort and planning.

The results were worth it. Not only did we surpass our target, but we also helped the business validate key assumptions and gain invaluable insights into their users and landing page experience.

While I love sharing these types of stories, I’d also encourage your audience to subscribe to the EXO Field Notes newsletter. It features interviews with top brands like Verizon, Nike, Microsoft, and LinkedIn, highlighting successful experimentation strategies and lessons learned. It’s free to subscribe, and I’m confident readers will find valuable takeaways from the industry’s best.

CRO Expert Profile Brian Poe

Cheers for reading! If you’ve caught the CRO bug… you’re in good company here. Be sure to check back often, we have fresh interviews dropping twice a month.

And if you’re in the mood for a binge read, have a gander at our earlier interviews with Gursimran Gujral, Haley Carpenter, Rishi Rawat, Sina Fak, Eden Bidani, Jakub Linowski, Shiva Manjunath, Deborah O’Malley, Andra Baragan, Rich Page, Ruben de Boer, Abi Hough, Alex Birkett, John Ostrowski, Ryan Levander, Ryan Thomas, Bhavik Patel, Siobhan Solberg, Tim Mehta, Rommil Santiago, Steph Le Prevost, Nils Koppelmann, Danielle Schwolow, Kevin Szpak, Marianne Stjernvall, Christoph Böcker, Max Bradley, Samuel Hess, Riccardo Vandra, Lukas Petrauskas, Gabriela Florea, Sean Clanchy, Ryan Webb, Tracy Laranjo, Lucia van den Brink, LeAnn Reyes, Lucrezia Platé, Daniel Jones, May Chin, Kyle Hearnshaw, Gerda Vogt-Thomas, Melanie Kyrklund, Sahil Patel, Lucas Vos, David Sanchez del Real, Oliver Kenyon, David Stepien, Maria Luiza de Lange, Callum Dreniw, Shirley Lee, Rúben Marinheiro, Lorik Mullaademi, Sergio Simarro Villalba, Georgiana Hunter-Cozens, Asmir Muminovic, Edd Saunders, Marc Uitterhoeve, Zander Aycock, Eduardo Marconi Pinheiro Lima, Linda Bustos, Marouscha Dorenbos, Cristina Molina, Tim Donets, Jarrah Hemmant, Cristina Giorgetti, Tom van den Berg, Tyler Hudson, and Oliver West.

Mobile reading? Scan this QR code and take this blog with you, wherever you go.
Written By
Brian Poe
Brian Poe
Brian Poe
Chief Optimization Officer @ Mammoth Insights.
Edited By
Carmen Apostu
Carmen Apostu
Carmen Apostu
Head of Content at Convert

Start your 15-day free trial now.

  • No credit card needed
  • Access to premium features

You can always change your preferences later.
You're Almost Done.
Convert is committed to protecting your privacy.

Important. Please Read.

  • Check your inbox for the password to Convert’s trial account.
  • Log in using the link provided in that email.

This sign up flow is built for maximum security. You’re worth it!