Overview of metrics questions in PM interviews

Metrics-related questions are frequently asked in New York product manager interviews. And being confronted with a question like “Y metric is down by X%, how would you figure it out?” can be scary. But don’t be scared; this article will outline typical metrics, give examples of queries, and explain how to approach them confidently.

Let’s first get rid of certain misunderstandings. There are many various kinds of metrics- 

• KPIs
• Feature level metrics 
• North star metric

Let’s briefly discuss some of the most important ones and their uses:

1. KPI: Let’s start with KPIs, quantifiable performance indicators used to monitor essential business processes. They frequently keep a check from quarter to quarter and display a firm’s future viability. For instance, a KPI for product utilization may track how closely the actual number of monthly users adheres to the target number (e.g., we want to maintain a 30-day retention rate of at least 50%). Monthly sales or revenue figures (for instance, we wish to reach $150K in daily income) are another potential KPI that is more of a financial nature. A KPI establishes a benchmark against which the team may gauge its success.
2. Feature-level metrics: The most frequently requested category by interviewers, feature-level metrics are commonly asked in metric success questions or product execution instances. If you were to introduce the ability to post videos on LinkedIn, for instance, how would you assess the performance of this feature? It is just one example of how one can question how you’d gauge the success of a new product launch. Another illustration might be this: Why do you think the amount of direct messages posted on Instagram has increased 10% day over day (DoD)?
3. North Star Metric: The North Star Metric is a single, ideal objective that the product leadership sets and that is recognized by all who impact it.

Facebook, which established the percentage of users who had ten or more friends in the first seven days, is the standard example in this case. They designated this as their “north star” after realizing that users were likely to become addicted if they connected with ten buddies in the first week (e.g., be a highly retained user). 

• Another illustration from a logistics team in the e-commerce sector may be OTIF: On-Time-In-Full delivery to customers, which indicates that 100% of orders are delivered timely without any errors or missing items. It also makes sense because the logistics staff is doing an excellent job if clients receive the right things at the right time! The team rallied around a hyper-specific statistic that has moved the company in the right direction using North star metrics.
• Metrics in the context of an interview: Why is the candidate’s familiarity with metrics important to interviewers? The short answer is that the hiring manager can assess a candidate’s mastery of essential product management tasks (such as determining whether a product is working or looking into a drop in critical metrics and usage) by looking at how they respond to questions about metrics. Standard metrics queries include questions like :

1. How and why would you evaluate the effectiveness of the new feature X?
2. Y metric decreased by X%; what could be the cause?
3. Which actions will you take to explore the recent spike in website engagement that you’ve noticed?
4. Assume you are in charge of PM for product X. What would be a decent North star metric to follow, in your opinion, and why?

A brief case study of the smartphone app “Clear Mind.”

Let’s talk about the metrics case study utilizing the hypothetical late-stage tech business that created the “Clear Mind” mobile app for measuring mental wellness. Users who want to exchange weekly progress reports and track their mental health days are the target audience for the mobile application.

Hundreds of users have currently downloaded and used the smartphone application. As a new product manager, you see that user retention has just decreased by 5%. It’s your responsibility to determine what caused this significant change (This falls under the KPI metric).

Focusing on retention is crucial for our startup to gauge the performance of its mobile app because it can be viewed as a proof-of-concept that users embrace by coming back to “Clear Mind” and spending time there. Here, the PM wants to conduct a systematic, transparent investigation to figure out what’s happening, and using a framework as a starting point is the ideal approach.

The following approach would be an excellent place to start when looking at retention or any other measure to identify the core reason. 

• Geography: Is the current retention decline unique to any location or area? Elaborate.
• Platform: For example, do “Clear Mind” consumers behave differently on iOS and Android?
• Channel: If the startup were focused on something other than mobile, it would be wise to compare app retention on the web, desktop, and mobile platforms. If solely mobile, do users gained through a particular channel have lower retention rates? (e.g., a poorly performing ad network).
• Market: Have any new, rival goods been introduced that target our user base and address comparable client needs? 

Any “yes” response from the interviewer to the questions above indicates that you should delve more into that subject. If not, you can delve deeper to find the primary cause(s) using a more product-centric framework:

• New capabilities: Has the application undergone any significant changes or UX/UI upgrades during the time in question? 
• Experiments: Are any experiments currently conducted on the “Clear Mind”?
• User base: Are there changes in the number of long-term (more than a month) or recently joined users (less than 30 days)?
• Data Analysis: Are we sure about data integrity’s trustworthiness, data sources’ validation, and confirmed methodology for analysis?

As we can see, asking the interviewer the questions listed above will assist prospective product managers in focusing on the case, coming up with suggestions for order success indicators, and determining which workstreams will be most beneficial to the startup business.

In this article, we examined metrics-based  Product Manager Interview Questions.

Concluding thought

 First, it’s critical to comprehend the various measures you could be asked about, ranging from high-level KPIs to north-star measurements. Secondly, planning and considering your response to typical metrics-related queries is beneficial. You can eventually create your unique framework for your answers, but feel free to use the examples above as a starting point.