Watch my Product School talk based on this post.
Introduction
Discovery
Discovery in the product world is the process of doing research for a problem area or product. There are many forms of discovery involved in product management and entrepreneurship.
Two well-known forms of discovery are product discovery and customer discovery. All things considered, one of the most important responsibilities of a product manager or entrepreneur is to fully understand the problem area and product being worked on.
The reason it is so important to have a deep and broad understanding of a problem area and product, is because a team can have the best executed solution the world has ever seen; however, if the problem is not correctly identified or understood, then the solution will likely miss its mark in meeting customer needs.
Methods for Discovery
In order to make sure your team executes with the most potential for success, it is of the utmost importance to conduct thorough research on your problem area and product.
This post will cover the following methods for discovery:
User Research
Stakeholder Research
Competitive Analysis
Product Analytics
User Research
According to usability.gov:
User research focuses on understanding user behaviors, needs, and motivations through observation techniques, task analysis, and other feedback methodologies.
User research is critical for discovery because it brings you in direct contact with the customer. This allows you to dive deep and understand the nuances of the user problems and processes that cannot be discovered through basic data.
When conducting user research, you want to make sure that you are talking to the right people to get the most learning. The best way to make sure you find the right people is by having clear criteria on the types of users that currently or might want to use the product. This traditionally has been referred to as a “persona” but over time is moving towards being defined as a “job to be done”.
Identify potential personas
The definition of personas from usability.gov:
[Personas] create reliable and realistic representations of your key audience segments for reference.
Effective personas:
- Represent a major user group for your website
- Express and focus on the major needs and expectations of the most important user groups
- Give a clear picture of the user's expectations and how they're likely to use the site
- Aid in uncovering universal features and functionality
- Describe real people with backgrounds, goals, and values
The definition of a job to be done from jtdb.info:
A Job to be Done is the process a consumer goes through whenever she aims to change her existing life-situation into a preferred one, but cannot because there are constraints that stop her.
The reason this article mentions personas and jobs to be done is because they are both similar practices with the intent of helping you as the PM or entrepreneur clearly define characteristics about the user.
Another way to define the characteristics of the target audience for your research is to simply use the 6Ws:
Who the user is
What the user is doing
When the user does this thing
Where the user does this thing
Why the user does this thing
How the user does this thing
For the purposes of this post, the following content will not pick any specific framework to use, but rather highlights the importance of understanding in total product or problem area.
Having a clear understanding of the 6Ws articulates the details of the user problem. It sometimes feels redundant or like overkill, but it can be the difference between building a mediocre product that fails to gain traction, and the next hit application.
In order to identify this criteria, it is good to start with initial hypotheses based on whatever understanding or research you have. This can come from prior experience, product intuition, third-party research, or anything else that seems fitting. The user research then aims support, reject, or build out these hypotheses.
There is a school of thought around not having personas/jobs defined before the user research; however, I have found this difficult because it then leaves you without a clear starting point. On the other hand, the risk of having hypotheses around personas/jobs too early is that you can get stuck going down a path based on incorrect data. It is important to fully vetted whether your initial hypotheses are valid or need to be adjusted.
Find and talk to people
Once you have a baseline of criteria for people who embody the characteristics that are relevant to your product area or product, you must go to find people to speak with.
For a product that you already have launched and have users for, it is good to use your own database of users and reach out to for interviews. If it is about understanding usage questions, make sure to query your database for active users of the specific product. For questions surrounding acquisition or churn, make sure to query new or inactive users from the database.
If you have a new product or new problem area that you need to talk to prospective users about, try finding similar or related products both internally and externally to establish a population of users to talk to. For example, if you are building a new local marketplace and want to understand the ins-and-outs of what a user needs for the marketplace, you might go talk to Craigslist users to understand their workflows. On the other hand, if you are a PM at an e-commerce site building a new tool for B2B sellers who currently have to use the B2C selling tool, you may go talk to sellers who use the existing tool to understand how it does and does not meet their needs currently.
Understand the nuances
When conducting user research, it can come in the form of both surveys and interviews. Generally speaking, surveys are best when you want to talk to a larger group and get quantitative data that helps you understand the problem. Interviews are best when you are interviewing a smaller group of people and want more qualitative data. The reasons behind these statements are that surveys are cheaper in terms of time and setup. They are also easier to pose numeric responses. Meanwhile, interviews are more expensive in terms of time and setup, but they allow for off-script questions that allow an interviewee dive further into details that the interviewer might not have thought to ask.
With that in mind, surveys are not limited to purely quantitative data, and interviews are not limited to qualitative data. Surveys can have open-ended questions that can allow the person answering to input freeform answers that give qualitative feedback. Likewise, interviews can be setup to record specific data points to gather quantitative data during the experience.
All things considered, regardless of the medium, it is critical to form questions in a way that allows you to dive deeper into the problem area to validate or reject hypotheses. Quantitative data should help support demonstrating the frequency or severity of a problem. A question can be asked, “How many times a week do you experience this problem?” to quantify frequency to help size the opportunity of a problem. Alternatively, a question can be asked “What do you do in order to address this issue?” in an interview to allow an individual to be more open-ended in helping the interviewer understand different ways a problem is handled in the current state of affairs.
All in all, it is important to leverage user research to understand the nuances of a problem with qualitative discovery and assess the prevalence, severity, or frequency of a problem with quantitative data.
Stakeholder Research
A stakeholder for your product or problem is anyone who is remotely tied to or affected by the product. Stakeholders include internal and external individuals. It is critical for you to identify stakeholders and consult them as a part of your discovery. They have a unique and differentiated perspective of the product and problem that neither the PM/entrepreneur nor the customers have. Creating or changing workflows for stakeholders can have serious implications for your product.
Identify potential stakeholders
Stakeholders come from a mix of internal and external partners. Some examples of internal stakeholders include:
Legal
Sales
Marketing
Support
Operations teams
Partner product and engineering teams
Whereas external stakeholders include:
Contractors
Clients of a service (which sometimes can overlap as the customer)
Identifying stakeholders involves listing any person, group, or entity that your product or problem area touches. This can mean that it touches them directly (such as when a business partner uses your product to execute on its goal)s or it can affect the stakeholder indirectly (such as an operations team that executes on the goal from the business partner using your product, and is by nature downstream from a direct stakeholder).
The best way to identify these stakeholders is to map out all of the entities that are affected by the problem you work on, or mapping out user flows for an existing product and identifying everything that the user flow affects and who is involved.
Understand stakeholder impact
Once you have identified the key stakeholders, it is crucial to understand how the problem affects their work.
This can take many forms. For example, changing your product could impact how the sales team pitches. Alternatively, approaching a new problem could open the company up to liability and would affect the legal team’s workflow.
In the end, truly understanding how your stakeholders are affected generally should come from interviews or emails to gather the qualitative assessment. If your product or problem area has tracking or data points that can be associated with a stakeholder group, this can also help to understand the problem area. Surveys generally do not capture all of the data necessary to understand stakeholder impact because these types of problems are frequently nuanced in nature which makes it difficult to capture them via a form input.
Gather stakeholder expertise
Finally, it is important to gather knowledge that any stakeholders have in the problem area or product you are working on. This almost always comes in the form of interviews, discussions or emails.
One example of this is interviews with the customer support team. This team is the closest to the customer and his or her issues. Support teams frequently tally how often a problem occurs and will propose workarounds to customers. This is a huge area of opportunity to gather knowledge on a problem area or product.
Another example includes discussions with the operations team. If you are working on a problem area or product that has multiple users, such as a tool to generate business content for the end-user, the primary partner might be a business team who generates the content, but they might employ a secondary stakeholder in the form of an operations team to execute populating and publishing the content. In this scenario, it is important to not only interview the business team who owns generating the content, but also the operations team doing the execution. These two stakeholders who are partners may have different and conflicting perspectives on the problem or product area.
Competitive Analysis
When thinking through the full extent of a problem area and product, it is critical to assess different competitors as you form customer problems and a product strategy.
The different ways to do competitive analysis include assessing:
Direct competitors
Indirect competitors
Well designed products
Poorly designed products
Direct competitors
Analyzing direct competitors is the easiest and most direct way to do research on a product and problem. It is generally pretty straightforward because your problem and product will have high overlap with the direct competitor.
A good example of this can be found with CRMs (Customer Relationship Management software). Compass has its own CRM product for Agents to use in order to facilitate workflows for managing relationships with past, present, and prospective clients.
When understanding the problem area for CRMs, the team tested other CRM offerings to understand what they did well and what they did poorly. In this scenario, two great direct competitors were Contactually (now a part of Compass) and Salesforce. Contactually was the top choice of CRM for real estate agents because it was light-weight and easy to loop in real estate-specific data. Salesforce, on the other hand is not used as much for real estate; however, it is the gold-standard for CRMs in the world.
When assessing both CRMs, the team focused on how the software fit well into user workflows and how it could be cumbersome or lacking in functionality that was critical to the agent relationship management model.
Aside from literally using the direct competitor products, it is also good to talk to users of these products to understand what their customer problems are and how the competing software meets these needs, and where it falls short.
All of this data helps to paint a picture of the existing market and what customer expectations are for your problem area and product.
Indirect competitors
Finding indirect competitors to reference can be a little trickier to identify; however, if you have clearly formed customer problems, they will start to become apparent.
An indirect competitor is an entity that represents a conflict between vendors whose products or services are not the same but that could satisfy the same consumer need.
Building upon the CRM example from above, some indirect competitors that further the research for a problem area or product include Google Sheets/Microsoft Excel and iPhone Notes. When assessing customer problems for the CRM, Google Sheets/Microsoft Excel met the customer needs for custom columns and filters whereas iPhone Notes offered the ability to quickly access and create freeform data that was easily accessible on-the-go.
Analyzing how potential or current customers use indirect competitors helps to identify gaps in your current problem statement and product. It also helps you break through your existing mental model of the problem and product to think bigger and more creatively about the customer.
Well designed products
Understanding the full extent of your product and problem area can extend beyond the competition to also include products that have great implementations and address their respective problems well.
As a product manager or entrepreneur, a micro-problem that comes up frequently in any experience is the concept of on-boarding. Doing broad research and discovery on a problem area and product involves understanding the different flows that will be a part of the product and how these flows have been accomplished for experiences that are completely unrelated to your domain. This helps to break out of the usual boring flows, get creative, and build better experiences than what is expected in your product area.
A strong example of this is the Beats Music app on-boarding experience. Not only has it outlived the app itself, but it also set a paradigm for future products. On-boarding is a tough experience to build because you have to not only collect information about the user, but you also have to explain the application and do all of this in a concise experience so that the user does not get bored and drop off.
The Beats Music app on-boarding experience was one of the first to move away from stagnant inputs and descriptions to be more interactive and game-like. When you started the app for the first time, you selected which genres that you liked, and the more you tapped a genre, the more you demonstrated that you liked it and the larger the genre grew. This came in the form of a visually-appealing interface that showed interests both as a scalar and relative input.
Poorly designed products
When assessing different products and problems during your learning, it is also good to note which products were designed poorly and did not address customer problems well.
A great example of this is the redesign Snapchat did for its navigation. The goal for the redesign was to expose Snapchat’s third-party media through the discover tab, but what it did was bury the most addicting feature that users craved, Snapchat Stories. Snapchat took a well-functioning and growing product and ignored user feedback to create a poor design that furthered business goals over user workflows.
Product managers and entrepreneurs can learn from these mistakes by making sure they do not replicate the same design paradigms, but more importantly, learn to not follow the process and steps taken that led to these poor decisions. For Snapchat, it was a result of poor research, poor testing, and poor monitoring. This example only furthers the argument that PMs and entrepreneurs should fully understand the problem area in order to make the right product decisions. Beyond this, PMs and entrepreneurs need to test and validate their decisions before fully committing to them.
Product Analytics
Analyzing product analytics is another great tool in understanding your problem area or product. Product analytics help to tell the story in a quantitative manner and can yield light on details that a user might not think to share during a user interview.
Digging into product analytics can come in the form of looking at current product usage, similar product usage, and different user flows/funnels.
Current product usage
If you have an existing product launched, understanding the problem area and product is as simple as looking at the tracking data. In an ideal world where key metrics have been previously established and tracking is firing correctly, looking at your metrics of success, metrics of failure, secondary metrics, and counter-metrics can help foster a deeper understanding of the problem area and whether your product is meeting the needs of the user.
Similar product usage
In a scenario where you are working at a large company that might have similar or tangential products to the one you have or are building, digging into these other products’s analytics can help you to establish a baseline, or it can be a comparison data set for understanding your product.
An example of this at eBay is the way the company represents listings for an item. There are multiple experiences that exist to represent a listing that have nuanced purposes. The first is the generic listing page that is the source of truth and standard representation of a listing.
A second experience to represent listings is one was optimized for Google PPC. Its goal was to surface a listing for Google Ads, and if the listing sold while the ads were still running, it would surface another listing in its place.
Finally, a new listing experience was built with the goal to group listings as products (i.e. have all iPhone listings from different sellers on a single iPhone product page). This was aimed to help facilitate shopping flows where buyers who are looking for an iPhone can land on a page just for iPhones that could handle all variations of iPhones from different sellers and help the buyer navigate to the specific listing that met his or her needs. As eBay iterated on its representation of a listing, it was able to leverage these similar experiences to aggregate analytics data to better understand the user needs more fully in order to build better experiences.
User flows/funnels
Lastly, analyzing product analytics for user flow/funnels is the perfect way to see what users do before and after they interact with your product.
For yet-to-be-launch products, these funnels help paint a picture of the full job the user is trying to accomplish. This can help you to see how the addition of your product affects the problems the user is encountering and help form hypotheses on how your work will best meet the user needs.
Wrap-up
Truly understanding a problem area and product requires a lot of digging as well as mixing qualitative and quantitative data.
It is critical to have a broad and deep understanding of the problem area. Your work might only focus on a small piece of the overall problem; however, it will still tie in to areas of the problem that you have actively decided to not address.
Researching the product and problem comes from a mix of direct and indirect research. Direct research comes in the form of user research for people who have or do use your product, stakeholders of your product, and digging into the analytics of your product. Indirect research comes in the form of talking to users of similar or competing products, analytics of tangential products, and competitive analysis.
Products evolve and develop over time. To be successful as a product manager or as an entrepreneur, you must have a passion and dedication for developing a rock-solid understanding of the problem area and product; in turn, you will be more successful in delivering the most value for your customers.