Ever been asked, “How would you evaluate the success of X product?”
It’s one of those classic “evaluate product health” questions, and trust me, it’s not just about knowing metrics.
It’s about showing that you can think like a Product Data Scientist.
Here’s the playbook I used to land offers when faced with this type of question:
Step 1: Understand the levers (and their priorities)
Step 2: Nail the metrics
Step 3: Slice and dice with segmentation
I’ll break down those steps, and then I’ll show you my step-by-step answer to this question.
Step 1: Understand the levers (and their priorities)
Start with the AARRR growth funnel.
Acquisition
Activation
Retention or engagement
Revenue
Referral
Your job? Spot the lever that matters most.
Ask these questions:
What lifecycle is this product on?
What’s the goal for this product?
What are the most important levers to pull, in order to achieve that goal?
For Facebook Groups:
It’s a mature product, so retention is king.
Monetization? Not the focus—no one’s spamming ads in Groups.
Growth? Already at scale, so we’re not chasing millions of new users.
Retention is the lever to pull.
Step 2: Nail the metrics
Metrics show the health of the product—but not all metrics are created equal. Break them down into:
North star: A single metric the team/company uses as a measure of success
Secondary metrics: They dive deeper into specific areas that influence the north star
Counter metrics/guardrails: Ensure things don’t go sideways while chasing growth. They’re like a reality check to prevent unintended consequences or quality drops
The key is not just listing metrics but explaining why they matter.
More on metrics in this article.
Step 3: Slice and dice with segmentation
Averages lie.
Groups might look healthy overall, but that could mask key issues. For example:
Android users may churn because of a UI bug.
Tenured users might engage more than newbies.
Segment by:
Demographics: Age, location, device, browser.
Activity: New vs. old users, super users vs. casuals.
This is where you uncover hidden trends and show you’re thinking beyond surface-level data.
More on segmentation in this article.
Putting it all together with my step-by-step answer
Let’s apply our grandfather framework, C-PODS, to crush this question:
“How would you evaluate the health of Facebook Groups?”