Why interviews are better than surveys
On the pros and cons customer research methods.
Various methods exist to gather information on your customers. In this article, I will explain the basics of why verbal methods are better at giving you broad insight than text based methods. It's for explorative, ambigious phases of any startup, innovation or design project. In my previous article I described this as the shaping phase.
An estimate per method, based on coaching over 50 startups and innovation projects. Do you agree?
I've ranked the potential insights per response for each method in the graph above. Potential insight means the potential amount of relevant information you can extract from a single response. Single response being one phone call versus one survey response.
Verbal methods—face 2 face (F2F) or (video)calls—have much richer information than non-verbal ones—text(email/IM). Intimate verbal communication settings allow you to ask follow-up questions, allowing you to dig deeper. Implicit communication such as intonation, body posture, facial expression give the depth that you need in the early stage. This implicit communication gives you cues to dig deeper.
Teams that are in close verbal contact with their customers shape their startup faster than ones that don't. The main point is about relativity. If you ask me whether you should get 5 survey results or 2 Face-2-Face conversation, I would say the latter. However, I would sometimes say that I'd rather have 2 video calls than 50 survey responses. This all depends on what you are trying to achieve, which I will address next.
When to do what?
When it comes to these methods, it's not about which one is better. It's about when—in which situation—each method works best. Below I will show you what I think are the relevant situations for specific customer interaction methods in an early stage startup. In this article I will not touch upon lean experiments such as landing pages and smokescreens. Those deserve an article of their own.
Low amount of insights per response
Survey
Great for:
Surveys are great for quantifying previously found results. If you want to know how dominant certain competitors are among a specific customer segment, a survey could show you that competitor A is used by 40% of the respondents and competitor B is used by 20% of the respondents. Why such information could be useful is a different topic. Sometimes I see teams that want to identify the reasons why people buy competitors' solutions. You would get answers such as "50% said speed, 40% ticked price, 20% reported tastiness". I feel that this information you could also get in an interview, without the values but with follow-up questions. "Why is speed important to you?"
Not so great for:
"You can add this why? question to the survey, right?", I might hear you think; I'm not a mind reader—If I were you could wonder why I would be spending my time on LinkedIn. You can obviously get open form fields with qualitative answers. However, the depth of these answers is often lacking. People don't generally enjoy doing surveys (high effort), so they might jot down the first two things that come to mind. These might not be the things that help you to reveal deeper unmet needs or identifying overarching jobs to be done. Most important thing: you can't ask follow-up questions. I've seen multiple teams start with a survey and switch the verbal communication methods for the early stages for above reasons. Students often reach for a survey because they are taught this is an academic way of doing research. However sound for that purpose, academic rigor will hinder your pace greatly.
Medium amount of insights per response
Email
Great for:
Email is a great way to get into contact with someone, new or existing customers. And as a way to keep in contact with them. As far as 'reaching out' goes, treating email as a gateway to meetings is advised. If a customer doesn't want to schedule a call to talk about its problem for 15 minutes, this might also be a red flag for someone that is not your ideal customer. Text based communication can be super useful if you quickly want to verify a thing or two. It's very dependent on the type of customer relation that you have, though. This works best for existing relations.
Not so great for:
With limited success I have seen teams mail a list of 5 (sometimes 10, jesus!) questions to various stakeholders. The most likely result is no response. If they respond, usually the answers are very brief. Emailing someone you don't know with a list of questions can easily be perceived as extra homework, nobody wants that. People like talking about themselves; this is something you should facilitate. You will have a higher success rate with one simple overarching question and a request for a talk.
IM
Great for:
IM (instant messaging, such as WhatsApp) works well for lengthier relationships with customers, especially consumers are open to this. I've seen multiple startups using WhatsApp as a customer touchpoint when launching prototypes. IM/Text is a great way to do small check ins. I often see startups create weekly surveys. The response rate on those things is quite low. For the customer, sending a few replies to personalised text messages is much easier. It yields better results as the barrier to respond is super low. Also you can immediately jump in on issues that arise.
Not so great for:
Quite some B2B, especially legacy industries, consider IM to be too informal. If you get your hands on a contact, don't text them. An introductory phone call is better.
High amount of insights per response
The following methods all fall under the umbrella term of interviews. Depending on your situation each of these methods can be relevant.
Phone call
Great for:
Phone calls are great for getting to know someone. Especially in B2B relationships, it's very common to have a verbal calls like this. Phone calls are great since you have the opportunity to ask follow-up questions. On top of that, you can tell a lot by the way a person speaks. Are the answers short or long? Is the person describing their issues in a pressing matter or something that he or she rather doesn't talk about?
A collateral gain of phone calls is that you start to build an intimate relationship with this person. Speaking out loud triggers a different experience for the customer than entering the same answer in a text field. The fact that a person is listening, with confirmations such as 'yes' and 'please tell me more', will open up a lot of extra details you will easily capture in email, IM or a survey.
Not so great for:
Phone calls are time consuming. This is suitable for early phases. If you have thousands of customers, this obviously doesn't scale well. Furthermore, you are missing out on visual clues.
Video call
Great for:
The video call is the upgrade of the phone call. All benefits of the phone call apply, yet you get to see the person's face. If you can freely choose between the video or phone call, I would start with a video call. This tells you even more. Be mindful, a video call might be a bit more intimate for some people. Sometimes it's best to offer both so the person can choose.
Not so great for:
A video call requires a bit more investment in terms of time and effort for the receiving end. Video calls require you to be behind a laptop, or at least a lot of people perceive it as such. If a person is busy, a phone call might open up opportunities to speak briefly, e.g. when this person is on her way home in her car. Also, in some scenarios a walk and talk instead of sitting behind your laptop can generate more honest results than an interviewy setting.
Face 2 face
Great for:
This option has all benefits from the two calls above. However, this option allows you to witness the customer in their context. If you can visit the customer when he or she is using a competitors product (or your prototype), you gain so much more insight. This applies for visiting companies too. You get a sense of the vibe at such a place. This is the most intimate way to connect with your customer.
Rule of thumb
Most of the time at the start of your projects, you want to have a high or very high amount of insights (see table below). Face 2 face, video or phone calls are your best bet. Depending on the type in information you are after, you can switch to other methods. Surveys are best if you can articulate what you are after and have a question of how much.
How was this article?
Great - Good - Meh
Any vote helps me, makes me smile like a baby seal in Hawaii.