How to Interview Customers with Teresa Torres – Masters of Product Management
Whether you’re on the road or away from your desk, you’ll want to stay connected with innovative ideas from Sequent. Our Masters of Product Management podcast provides listeners with engaging topics to help you think differently and about things you encounter daily, but often don’t have enough time to reflect and take action. Get your dose of insights, in just 15 quick minutes by tuning into the Masters of Product Management podcast.
In order to ensure that a product’s features hit the mark, product managers and their teams must become exemplary interviewers. This is a vital skill for teams who have to be inspired to continuously learn and fine-tune the value of their products. In this episode, Steven Haines speaks with Teresa Torres from Product Talk to inspire you to be great customer interviewers. Hosted by Steven Haines of Sequent Learning Networks.
Key Points:
-
Though product managers need to spend about half their time interviewing customers, most do so a lot less, if at all. This means they can’t discover their customers’ needs.
-
When conducting interviews, product managers need to be cognizant of their confirmation bias, which can overshadow criticism they receive.
-
It’s important to have people from different fields present at interviews, because we all have different biases and perspectives. An engineer, for example, will hear valuable information that a product manager might not pick up on.
Transcription:
[00:00:28] Steven: Welcome back, everyone, to Masters of Product Management. So, if you asked 20 product managers if they interviewed customers, to either uncover customer needs or validate a feature set within a user story, or anything like that—or any motivation, they would all say, “Of course we do!”
Then, let’s take it one step further. Ask each one of them to describe the technique that they use, and I will put money on the fact that you’ll probably get about 20 different answers.
And let’s peel this back one more layer: find out or try to discover what was developed, or conceived, or envisioned, based on those interviews, and whether or not what they created hit the mark for those customers. And you will find a host of different responses. And, I’ll bet you’ll get less than satisfactory outcomes.
[00:01:28] Steven: Now, you’re all busy. You’ve got lots to do. I’m going to share with you an interesting tidbit of some preliminary research that I’m doing, and that is whether or not product managers actually are getting out, or doing enough customer interviews.
And roughly—there is a belief that they should be out and about, about 50 or 60 percent of the time. And then one of the other questions talks about how much you’re actually doing it, and how much are you actually conducting those interviews, and it’s about 20. And I think it’s even less. So, stay tuned; I’ll have some more information on that shortly.
[00:02:03] Steven: Anyway, wouldn’t you love it if your user stories were accurate and developed at the right context that actually delivered a feature with an experience or value prop as what you envisioned? And that’s, I think, the challenge; and that’s really the topic of the show today: it’s how to interview customers.
And today, I have the honor of talking with a really wonderful human being—a classic professional in the field. Her name is Teresa Torres. And she’s going to help us with some tips on how to be a better interviewer. And so, Teresa, welcome to the show. Thanks for being here today.
[00:02:41] Teresa: Thanks, Steven! I’m excited to do this.
[00:02:43] Steven: Yes. So, I’m not going to read a bio; could you tell our audience members a little bit about who you are and what you do?
[00:02:51] Teresa: Yeah! So, I work as a product discovery coach, which is kind of a mouthful! It basically means I help product teams—so, product managers, designers, software engineers—figure out what they should be doing. And we do that through customer interviews, rapid prototyping, and product experiments.
[00:03:09] Steven: That’s very cool. And I love the topic of experimentation; I hope you get a chance to talk about that a little bit. Now, we’ve spoken previously about this, and the intro talks a little bit to the point—and it’s maybe leading the witness, but can you talk to the audience a little about what the heck is going on, and why this is so important for product managers, and things that you may see that are problematic?
[00:03:36] Teresa: Yeah. I think the hard reality is that we still have a lot of product teams who don’t interview their customers. It’s getting better, but there are a lot of teams out there that I still meet, that don’t talk to their customers ever; or they don’t talk to their customers nearly enough; or they talk to their customers in the context of a sales call, or in the context of an account management team.
Now, we do have other teams that are getting better at this. They’re doing customer development interviews; they’re talking to their customers just to uncover unmet needs. But the challenge I see is that even when we do interview, it’s a skill that needs to be developed.
So I will always tell anybody: you’re better off talking to customers than not talking to customers, even if you have no idea what you’re doing. But I do think that if we invest a little bit in developing the skill, we get a lot more reliable information back from our interviews.
[00:04:31] Steven: That’s a really cool thing. So I’m going to put this on my little notepad about the skill to be developed. But as you were talking, you talk about how teams interview customers.
And when I talk to product managers, it’s not that they have this whole army of people who are going to go out and do listening; but in my own career, when I would do onsite visits and interviews and observations, I’d have at least two or three people with me, because I don’t see things other people see, which is still the case in my life. But who are these people, and how do they get throttle to participate in this?
[00:05:10] Teresa: So, when I say “Product Team,” I usually mean a product manager, a product designer, at least one engineer. With interviews, obviously, the more people you have in the room, the more uncomfortable the interview participant is going to be.
I believe technology to help address that, right? You can have people listening that aren’t physically in the room. And I think the key is that those three perspectives need to hear about the customer’s experience.
I think our background knowledge and our background experience really impacts what we hear in an interview. I think in most discovery activities, we want all three of those perspectives participating as frequently as possible.
[00:05:52] Steven: Yeah, I think that’s really important. You know, I recall—I used to work for Oracle, and I was involved in business process automation. But the people who I would both visit customers and observe with, had different skill sets.
And I think it’s because of our bias that we have, and the paradigms that we have, because I see things through a process lens; not necessarily through a design lens. And having the user experience person with me, or with a product team, is a really important thing—or a designer, as you would say. Correct?
[00:06:25] Teresa: I think the same is true with engineering. So oftentimes, a customer—because they don’t have a lot of knowledge of what’s possible with technology—might talk about something that a product manager or a designer would just dismiss because it seems too pie-in-the-sky, while that might actually motivate an engineer to go figure out how to do it.
[00:06:41] Steven: Yeah, that’s true as well. Okay, so we have the team down. We have to have a group of people who can both observe and shear and process from their perspectives; so that gives us one anchor point.
But then we talk about this skill that has to be developed. What is this skill—this interviewing, and anything else that’s surrounding that particular context?
[00:07:09] Teresa: The first thing I’ll share is that, I think, if you’re not learning surprising things in your interviews every week, your skill probably needs to be developed. I hear all the time from product teams that say, “Yeah, we do interviews. We’re validating our ideas.”
And that’s great.
We do need to validate our ideas; but if all we’re hearing is that we got it right, odds are we’re falling prey to confirmation bias. And the funny thing about biases is they’re becoming really trendy. We all know about them. But there’s this thing about biases: just knowing about them doesn’t mean we don’t fall prey to them.
[00:07:47] Steven: [laughs] Wow, that’s really good! So I know I’m biased, so let’s just exploit that and celebrate that!
[00:07:52] Teresa: [laughs] We all think we don’t fall prey to the bias; so our natural inclination is, “I got everything I could out of this interview!
I’m not subject to confirmation bias!” Whereas reality is probably—unless you’re guarding against it from the get-go by the design of your interview, you probably are falling prey to your confirmation bias.
That’s why we hear great things in interviews; and then we build stuff and it doesn’t—it’s not quite as great, if we’re being honest with ourselves. It’s not quite as great as what we heard in that interview.
[00:08:23] Steven: I’m going to ask you a question, because sometimes people in the audience may not have all the vocabulary words. Could you describe what confirmation bias is?
[00:08:31] Teresa: Of course! So, it’s the bias where were twice as likely to hear confirming information than disconfirming information.
So, if I have a great idea for a solution, then I’m asking an interview participant about it. I am twice as likely to hear—forget “pay attention to.” Forget “follow up on.” Just hear—literally, hear the positive feedback—twice as likely to hear the positive feedback that I am to hear the negative feedback.
[00:09:03] Steven: You know what that’s like? It’s like what my mother used to tell me! [laughs] She said, “You only want to hear what you want to hear! [laughs].
[00:09:11] Teresa: That’s exactly right! In fact, I have an article that I wrote about confirmation bias and it’s called, “You Only Hear What You Want to Hear.”
And I think this is exactly why it’s not enough to just do interviews. We got to get over the hurdle of, how do we do an interview that accounts for, “We’re going to hear what we want to hear”?
[00:09:30] Steven: That’s great. Okay. So, I want to go to that in a second, so park that in your head. Now, you said, “You should be learning new things every week.” Now, how many product managers can actually be doing interviews every week—and are they really doing that?
[00:09:51] Teresa: Yes. Most teams that I work with are terrified when I tell them that I want them to interview a customer every week. This is a terrifying idea; I get it.
[00:10:01] Steven: I hope not.
[00:10:01] Teresa: Sadly, it is. I’ve worked in companies where the idea of interviewing is once a month, or once a quarter. Or some centralized user-research team does that for us.
But here’s the reality: we make product decisions every single day. And the longer we go between interacting with our customers, the longer we go perhaps making the wrong product decisions.
So, I like to think about this as how do we at least on a week—at a minimum, on a weekly cadence? And frankly, I think the best teams are talking to their customers way more often than weekly.
But it doesn’t have to be this formal, hour-long three-page discussion guide…set up a dual—a room with a mirror—like, formal interview, right?
I’m talking about—just get out and talk to people; but do it in an intentional way, that accounts for your biases and allows you to get reliable feedback.
[00:10:54] Steven: You know, we have a couple of different dimensions. One could be a scrum team that’s doing a two-week sprint. They should be able to have this stuff set up—let’s say they’re doing their stand-up on a Wednesday or a Tuesday. They should be able to say, “All right.
We’re going to go out by Friday and have a chat with X customers who we’ve got lined up, and we’re going to report back on the following Tuesday or Wednesday,” whatever that day is.
Wouldn’t that be something that you would want them to be able to do, so it’s on their list?
[00:11:22] Teresa: Absolutely. In fact, I think the key to doing a weekly interview is to automate the recruiting process. So what my goal when I worked with teams is, they show up to work on Monday and have an interview on their calendar without them having to do anything to get it there.
[00:11:38] Steven: That’s great.
[00:11:38] Teresa: An interview is just the same as showing up to a meeting, right? We go to dozens—most product managers are triple-booked; they go to dozens of meetings a week.
They easily could make room for—let’s just do a 30-minute interview, if it was put on their calendar for them.
[00:11:57] Steven: Right. Now, part of this is, you might want to show them something, or at least—them meaning the customer—may want to at least tell them, or show them, or give them something to respond to. So the object of the interview can be transient. Is that correct?
[00:12:13] Teresa: Yeah! So I think—I like to use interviewing for two purposes. One is just to learn about my customer: tell me about your day; tell me about the way you do X.
Tell me about last week, right? Just raw and open need-finding. And then a secondary part of the interview is, I have an opportunity in mind. I have a need in mind that I want to explore. I might move from something like, “Tell me about your day,” to “Tell me about the last time you did this specific thing.”
And then, of course, I can take that one step further—if I have a prototype, I can get feedback on that prototype. But again, if we make product decisions every week, then every week my questions that I want to get customer feedback on should be changing.
[00:12:55] Steven: Yeah, that’s true. And I like the idea of these open-ended questions. You know, we always talk about that stuff, and I think that even in my own—I tend to, you know, I want to find out more. I want to confirm.
So, I set myself up, and I might ask the question that is only half open-ended because I want them to respond. But there has got to be an art to the open-ended question. Could you help us—even help me—understand how to do a better job with that?
[00:13:21] Teresa: Yeah. I think the biggest mistake teams make with interview questions is they think about what they need to learn, and then they ask the participant those questions. There is no translation between “Here’s what I need to learn” vs. “This is what I’m going to ask to learn that.”
I’m going to use Netflix—pretty popular product—as an example. If I work at Netflix and I need to understand my interview participants’ viewing behavior, my inclination is to ask things like, “How often do you watch Netflix? What do you like to watch?” Maybe, “Do you binge-watch?”
This is what I want to learn; but it’s not a great way to ask the participant. Because, Steven, if I ask you how often do you watch Netflix—first of all, you may have never stopped to think about how often you watch Netflix; which means your answer isn’t much better than a guess. There are other things that can come into play, right?
Maybe you don’t really want to tell me how much you watch Netflix because you’re embarrassed by it. So there are a lot of challenges with these direct questions.
Instead, I think what we want to do is we want to think about–I don’t want my participant to ever speculate in an interview; so I don’t need to speculate about how much you watch.
I don’t want you to speculate about what you like to watch. I want my interview participant to share specific stories about their past behavior. So, instead of asking you how much Netflix do you watch, I’ll ask, “Steven, tell me about the last time you watched Netflix.”
[00:14:53] Steven: Like, “I was watching it and then I got interrupted and then I lost my place. And then I went and did something, and then I didn’t remember what episode I was on.” That kind of stuff?
[00:15:02] Teresa: Yeah!
[00:15:02] Steven: And then I got distracted—I mean that you want to know the behavior.
[00:15:04] Teresa: I want to know the whole story. I want to know where you were, I want to know what triggered it, I want to know what you were trying to accomplish.
I want to know what device you were on. I want to know if it was a good experience. And this is something that—this is where I think the art of interviewing comes in. So much of a conversation is this 50/50 back-and-forth. I say something, you say something, I say something, you say something.
So, when we ask someone for a story, they tend to tell the most abbreviated—like if I just said, “Tell me about the last time you watched Netflix,” you’d probably say, “Oh, I was watching four episodes of Breaking Bad on Sunday.” And that’s not a very good story.
So what I would say is, “Well, Steven, tell me about Sunday. What were you doing right before you decided to watch Netflix?” Great. Now talk me through it. I want all the detail. And odds are, I’m still going to have to prompt you. “Okay, hold on, back up. Then what happened?” Right?
And not only am I going to learn a lot about your Netflix habits; I’m going to learn things like what device you like to watch on, what types of shows you like, in the context of a real story. I’m also going to learn all the nuances and the context in which you’re using my product. And then I can do this multiple times, right? So, I can say, “Tell me about the time before that.”
[00:16:12] Steven: I have a challenge to this, then, because my brain is abuzz now. I want to be on the line with you for, like, an hour and a half now! But now, I have been accused of not remembering all the details of a situation—like, “You did this and what happened?”
And they end up coming out over periods of days. So somebody actually asked me the question about how to describe that experience.
I’d come up probably blank, and you’d probably get really frustrated and beat me with a bat. Are there people who make good interviewees vs. bad interviewees?
[00:16:50] Teresa: I think maybe more of what’s at issue there is timing. I don’t want to ask you about something you did last month, right?
That’s not going to be very reliable. This comes into play with recruiting. If I need to interview somebody about their Netflix behavior, thankfully I can look in my database and see who’s watched Netflix recently, and interview those people.
And even so, you may still not remember, in which case I just need to move on to another type of story, right? I don’t want to force you to remember, because now you’re likely to make things up.
[00:17:24] Steven: So the interviewing team—getting back to the team, has to have—they need to know who this person is, and they need to have some context around what it is that they’re looking for, and perhaps even have rehearsed a little bit beforehand.
[00:17:41] Teresa: Yeah! Preparing for the interview is critical, right? We tend to think about preparing as writing a three-page discussion guide; and all that does is it makes the interview about what you want to learn, instead of about what the participant wants to share. And if we want reliable feedback from the participant, we have to care about what they care about.
[00:18:02] Steven: Ah!
[00:18:02] Teresa: And so I think, instead of writing three-page discussion guides, we should spend that time learning about—what do we have in our database about this customer? How do we recruit them? What do they care about? What do we already know today? And you shouldn’t spend any interview time asking them things about what you already know; and instead, collect those stories.
[00:18:24] Steven: Wow. This is so incredibly interesting, and it’s very exciting. And I’m looking at my clock over here because I already went longer than I said I was going to go, because I’m way too interested! And what are you gonna do.
But this—you have brought up so many very interesting and thought-provoking ideas, and I think even the fact that we’re having a conversation almost is a reflection for learning and understanding how product people can be more effective in their jobs, and how to do a better job of interviewing people.
But if there’s one thing that I’ve learned, Teresa, and it’s that this is practice. The product manager and the product teams need to do this on and on and on. And this idea behind doing it every week produces a discipline, or induces this discipline, that I think really would lead to better practices and better outcomes.
So I think that’s really outstanding. And I wanted to thank you, really, for taking your time and being on the show today, because—again—this is inspiring and insightful, and it’s something that’s very, very important for product managers to think about.
[00:19:41] Steven: And I’m going to leave them—I’m going to leave people in our audience with a thought: this notion behind, “I don’t have time,” and “I’m going to too many meetings,” or “I don’t have any budget,” is not acceptable anymore. The world has become so incredibly competitive.
And I’ll go back to something: in my product management workshops, when I ask people how often are they talking to their customers—how often are they interviewing or visiting—and we see less than 10 percent of the time. How do they know that what it is that they’re working on will actually hit the mark? And it goes back to what I did when I opened up the conversation today.
[00:20:22] Steven: So, that said, thank you so, so much today, Teresa. It was really amazing, and I hope that we have a chance to talk in the future about some of the things that you’re working on.
Thank you very much for that.
[00:20:34] Teresa: Yeah, absolutely! Thank you, Steven.
[00:20:35] Steven: You’re quite welcome. And for you in the audience: if you have a topic you’d like me to discuss, or you’re interested in training, coaching, or other product management resources, please do reach out to me through the contact form at Sequent Learning Networks, or through LinkedIn or Twitter, or anything like that.
And remember: you can always find these podcasts and refer to your friends on iTunes and Stitcher, and all those great resources. Again, thank you so much for listening in and we’ll see you next time on Masters of Product Management.