The Way We Do Customer Discovery is About to Fundamentally Change
If you routinely interview customers before building a new product or rolling out a major feature, your process might look something like this:
Kickoff a study
Plan to interview 10-15 people over two weeks
Run interviews / take lots of notes
Circle back with the team to make sense of your notes
Look for patterns and prioritize problems worth solving
Formulate possible solutions, test, build, measure, learn…
So what’s wrong with this picture?
The challenge with any kind of qualitative learning is prematurely jumping to the wrong conclusions. Why does this happen?
Here are some of the top reasons:
Unless two people run the interview, it’s hard to both lead the conversation and capture all the insights as notes.
Even if there are two of you, there isn’t a standard way to take notes, which makes them subjective and prone to the worldview and biases of the note-taker.
If there’s a disagreement on a particular insight, there is no way to audit the conversation.
And these are just the challenges with capturing insights after an interview. When we attempt to mine these insights for patterns, new challenges emerge:
We unconsciously look for evidence in the notes to validate the problem/solution combo (aka feature) already in our heads. Yes, this is our Innovator’s Bias at work.
Apart from the Innovator’s Bias, we easily fall prey to recency bias and other forms of confirmation bias, where we pay disproportioned weight to parts of the conversation that align with our already-formed worldviews.
The cost of prioritizing the wrong problem is wasting needless time, money, and effort building something only to realize a meh response post-launch.
Prioritizing the right problem(s) worth solving is too critical to leave to chance.
Earlier this year, we set our sights on improving the current customer discovery process at LEANSTACK.
Here’s what we came up with.
Part 1 of the Solution: Transcription
Part 1 of the solution was simple: Record and transcribe all your interviews. Transcribing an interview used to cost $1/minute 15 years ago. Today, you can transcribe a one-hour interview for $1. That’s a 60x improvement!
Most customer conversations already happen virtually, making it relatively easy to record the conversation (with permission, of course).
Some obvious advantages of transcription are:
It relieves the pressure of having to take great notes in real-time.
It captures the single source of truth (aka the voice of the customer), which makes your insights auditable.
As it can be heard/read by your entire team (accessible), it helps keep individual biases in check.
Some non-obvious advantages of transcription are:
It can be consumed much more quickly as we read faster than people speak.
But what I’m most excited about is that it can be mined much more deeply, resulting in more actionable insights.
One of my favorite features we tinkered with was utilizing machine-driven sentiment analysis to score and visualize an interview. Sentiment analysis works by scoring what the customer said into positive, neutral, and negative statements.
So something like “I like shopping at that store” scores positive, and something like “I hate pink” scores negative.
We use these scores to visualize the emotional shape of the conversation and to compute an overall sentiment score. For more on this, see The Simple Shapes of Customer Stories.
To be clear, all this happens automatically (machine-driven), and less than five minutes after uploading an interview, you get a picture that looks like this:
The first place I focus on is the overall sentiment score. Does it correlate with my assessment of customer satisfaction? If not, why not?
The visual shape of the conversation helps in pinpointing high and low moments in the conversation (emotion or energy) for deeper study. This is often where one finds spaces for innovation.