Testing Mind Map Series: How to Think Like a CRO Pro (Part 19)
Interview with Tim Mehta
CRO veteran Tim Mehta, Director of Growth Marketing at Lucid Software, has seen firsthand how CRO can benefit more than just revenue growth.
Tim has spent the better part of the last decade building and improving experimentation programs and teams. On the agency side, he built test & learn programs for a range of businesses from scratch. At places like Microsoft, he helped optimize testing programs already high in maturity.
Tim says revenue should not be the end-all-be-all goal for testers, but rather one (albeit important) metric by which to measure success.
In other words, the benefits of CRO are far-reaching and can have a positive impact on every area of your business.
But despite its clear benefits, implementing effective CRO strategies remains a challenge for the average company.
In this post, we’re chatting with Tim to explore some of the reasons why companies struggle with CRO and offer some tips for making the process easier.
Tim, tell us about yourself. What inspired you to get into testing & optimization?
My background really started in marketing strategy and morphed into analytics, then testing. I couldn’t consciously tell you why, but I just found myself most interested and excited about UX and testing. Every time I’d come across a book, podcast, or article about it, I was hooked in.
As far as inspiration, it’s really been through interacting with other testing colleagues who were smarter than me and have been doing this a lot longer.
How many years have you been optimizing for? What’s the one resource you recommend to aspiring testers & optimizers?
I’ve been optimizing for just over 7 years now. The best resource in this line of work will always be other experimenters. Connect/follow them on LinkedIn, listen to them on podcasts, watch their video conferences, even chat with them at weekly video meetings (like this one).
The next best resources would be
- Reading books about experimentation and product management.
- Reading Medium articles about testing (most of the ones I find are about statistics, which is super useful).
Answer in 5 words or less: What is the discipline of optimization to you?
Customer informed decision making.
What are the top 3 things people MUST understand before they start optimizing?
- Basic statistics/analytics.
- You (and everyone else) will be wrong 80%-90% of the time.
- How to communicate the scientific process in a simplified way.
How do you treat qualitative & quantitative data so it tells an unbiased story?
Qualitative data must be focused on identifying pain points and validating usability. Using qualitative data to ask people what the solution should be is risky because, when tested in the real world, those same people will behave differently than they say.
It’s still important to source customers for potential solutions, but those ideas should always be vetted and tested just as much as our own “gut feeling” solutions.
What kind of learning program have you set up for your optimization team? And why did you take this specific approach?
Each team member should have a unique learning plan. Some may need more help/training in statistics, some in UX, some in communication or relationship building.
Each person has a unique learning style. My job is to find the knowledge gaps, understand the learning style, and find/provide the right tools to help.
Other than that, I will often recommend podcasts, articles, or books about the subject that I gained value from but try not to pressure them to consume it. The best learning plan is when they follow their own curiosity and I just feed it.
What is the most annoying optimization myth you wish would go away?
The myth that the only value of a testing program is “Revenue gained.”
Gaining revenue is a part of what you can do with experimentation, but the best testing programs provide intangible value well beyond that.
Thanks to Tim for sharing so many valuable insights! Save an abridged version of the interview to your swipe file to refer to at any time.
Tim had a lot of great things to say, and we hope that his insights will help guide your experimentation strategy in the right direction.
What advice resonated most with you?
Keep an eye out for our next interview with a CRO expert who takes us through even more advanced strategies!
And if you haven’t already, check out our interviews with CRO legends Gursimran Gujral, Haley Carpenter, Rishi Rawat, Sina Fak, Eden Bidani, Jakub Linowski, Shiva Manjunath, Deborah O’Malley, Andra Baragan, Rich Page, Ruben de Boer, Abi Hough, Alex Birkett, John Ostrowski, Ryan Levander, Ryan Thomas, Bhavik Patel, and our latest with Siobhan Solberg.
