Product Discovery for Data PMs PART 3 - Continuous Discovery for Internal Tools

In this substack, we are exploring (a) the concept of continuous product discovery and (b) how can you apply that to an internal product.

Hello Fellow Data PM enthusiasts.

Welcome to the last and the final part of the “product discovery” series. I promise this one would be different, as it doesn’t focus on “end consumer” facing product, but on internal discovery. And, doing product discovery becomes even more important for “internal products,” and we often ignore a structured approach.

Continuous Product Discovery

But, before we dive into Case 3, let’s recap the most valuable concept in product discovery today: “Continuous Product Discovery.” CPD was made famous by Teressa Toress, and she set the foundation of what each product team and product manager aspires to champion today: doing product discovery well. The idea is to continuously discover new product ideas systematically and create new backlog items that can then be built into the product.

Jeff Patton’s take on dual-track.

With CPD, a PM aspires to discover new ideas by spending weekly time to discover new opportunities with customers and thinking about something called an “Opportunity-Solution Tree” (see below). The idea is that your product is a living/breathing thing, and you are NEVER done. PMs should create new product opportunities, think through multiple solutions for each of these opportunities, as well as run experiments to validate assumptions.

You can use customer interviews, prototype testing, usability testing, content testing, or simply watch sales/internal calls as a means of customer discovery.

Here are some of the best practices around CPD:

1. Keep prioritizing new opportunities as well as growing your understanding of the customer/user base.

2. You shouldn’t over-index to the goal of the customer, balance it with the goals of the company.

3. Avoid assumptions/biases of internal folks, by having a counter information from the customer or any other persona.

4. While you are gathering user pain points, gains, and use cases, don’t ignore getting input from Engineering on “feasibility” and business “viability” from other GTM motions.

5. Don’t use customer discovery as a means for validating your biases.

But, why am I talking about CPD when talking about “internal tools”?

Because we often ignore internal users and don’t treat them with the same level of importance as end customers, and as Data PMs, most users can be internal. Therefore, doing continuous product discovery becomes even more important with internal tools to generate reasonable value for the business.

Case 3: Doing product discovery for internal tools.

Internal tools help us drive the cost of operation down. And, as a Data PM, building good internal tools to support your dev/other teams to build great products is quintessential. Also, picking up an internal tool project gives you a chance to meet with “all your internal consumers.”

So, for this week, let’s discuss discovery for internal tools that have “internal teams” as an audience. Let’s look at an internal “observability” tool for a data platform, but before that, let’s discuss how discovery is different from a customer-facing product.

How is discovery for internal tools different?

1. Well, the way discovery for internal tools is different is that your coworkers are your “customers.” While getting access to the “end user” for discovery is easy, there are other things such as team dynamics, being careful about how the discovery is perceived, and whether you are over-promising or not.

2. A huge downside, you might not have pre-existing data or signals that you can rely on for your product judgment. So, either you have to do lots of interviews, or you have to get data out of internal systems such as Jira, Slack, and other team collaboration tools to make objective decisions.

3. It’s not going to impact “revenue” as a metric, maybe affect “cost” at best, so getting funding for such projects can be harder or such products can get deprioritized unless someone higher up feels the pain.

Case in point

Let me now talk through a specific “observability” project from the past: what were my constraints, how did I apply CPD, what did I do well, and what did I not do well?

My constraints

1. I had limited knowledge about the product area and the company, as this was one of my first tasks. I had no idea about how we did any observability before.

2. The project was in-flight and I had to onboard quickly, and we were beginning our planning cycle for the next 3 months.

3. Like any team, we had multiple different tools that we were using or could have leveraged, so design decisions about thinking long-term were tricky.

How did I approach product discovery?

Since I didn’t have much context, I started by building my context and getting started on what was going on in the current sprint, as part of the current plan, and understood the requirements and constraints. Here’s how my discovery went:

1. I first spoke to a few senior people to reconfirm the outcome we were trying to drive and define the “outcome” that we want to drive from this effort.

2. Each week, I tried to interview a new persona to understand various takes on the problem statement. Week by week discovered “opportunities” continuously, and kept prioritizing “opportunities” that had higher ROI for the outcome. These interviews also gave me the confidence that the discovery wasn’t over-indexed to a specific person/team.

3. For validation of the problem space — I spoke to a wider audience, who would not directly benefit, but indirectly will benefit.

4. I also regularly checked in with my Engineering counterpart on our design of the tool so that there was a continuous check on the “feasibility” risk as well as the “viability” risk.

5. Simultaneously, I also interviewed products that were solving the same problem in other companies or external tools to understand if we should pursue a “buy” vs. “build” path. Also, created a map of what information is necessary for observability of a data platform to have a more holistic vision.

6. Over time, we found many opportunities and were also able to define what “observability” would mean tangibly. Had we started with defining the vision, and waiting to “deliver” any product, we wouldn’t have got anything out of the door, and might not also have a more confident vision.

7. This resulted in an Opportunity Solution Tree (OST) like the one below and gave us a roadmap of projects we can invest in. Some projects were no-brainers (from the interviews), others we verified by collecting data from internal tools, or by interviewing folks to estimate effort and return on each project.

Things I did well

1. Prioritized the most impactful project for the “outcome” and worked to put that in production, while evaluating other “opportunities.”

2. Spent time understanding the ROI, feasibility, engineering design, and viability of most opportunities so that we utilize our resources well.

3. Not everything needs to be built by devs, so involved other teams such as Ops and Project Management teams to define new processes/tool-based automation, which would impact the outcome too.
   

Things I didn’t do well

1. The classic mistake of not parsing through the interviews in a more structured manner. I could have collected the notes from different interviews in one place, which would help others track back why certain projects made more sense. Luckily, the team was aligned too as they had similar conclusions.

2. I could have taken time to build an “interview plan” and got that validated from someone senior so that I was clear on the audience I was going to speak to.

🔗 Links of the week

1. Evolution of modern product discovery by

   Teresa’s Substack
   . MUST WATCH!

2. How to get CPD in practice? Here’s a resource by userpilot.

3. An alt-take on whether you need Databricks/Snowflake or not by

   Kieran Healy
   .

That’s all folks! Hope this series was useful to get you started on Continuous Product Discovery, and with Opportunity Solution Tree, maybe you can map out how your work will or will not impact the larger company OKRs/objectives.

Signing off,

Richa

Chief Data Obsessor, The Data PM Gazette