This blog post was written by Mozart Data Co-Founder and CEO, Peter Fishman.
It’s quite common for an executive to want to know how an activity/feature/experiment is going. A typical answer reports on the successes (or success rate of that activity) frozen in a moment in time. I might ask, how is our new outbound campaign going – and a rep might answer, “Great, we’ve booked 14 demos.”
This is what I wanted to know and what I asked for, but it’s actually not actually what I want to know. Because insights from a moment in time miss the nature of the activity, which often plays out over time. On Sundays, I might make a bet on an NFL game; by Monday, a coworker can ask me, how did the weekend go and I can tell that person (typically not well), and that would be perfectly informative of the outcome.
Most of the actions my GTM team takes or features our team builds do not work like that. They take time to resolve.
So when I get an answer like we’ve booked 14 demos, it’s often misrepresenting what I want to know – it’s typically the demos booked in a given time period, rather than the demos booked from the actions I’m interested in.
Imagine I send 100 emails to prospects each day. If I book 5 demos per day, a natural inclination is that there’s a 5% success rate. But what I most often want to learn is – how did that recent set of 100 emails perform. The 5 demos booked may have resulted from the 100 emails I sent recently, or they may result from emails sent weeks or months ago.
This isn’t rocket science, it’s cohort analysis. It’s obvious of course that you want to be able to tie successes back to the activity, but it’s also sometimes painstaking or difficult to do.
To understand how a campaign is going, I need to understand how it has gone for cohorts that have neared completion; not just a sum of the recent successes. If there are good signals for total success early on in the cohort, that too can be used.
Creating datasets that are flexible enough to sum the present success, but also to evaluate success by cohort (starting time or other characteristics), is critical for finding insights about what is working. A pre-baked report can cut through an initial cut or two, but flexibility of the data set to answer the next question is critical.
