Customer development surveys tend to have a bad reputation nowadays: they’re known as costing a lot of time to create, and giving confusing answers at best. But surveys can be useful, if you design them in the right way and use them for the right purpose.
Here’s a list of the top mistakes people make when building & using surveys, and what to do instead:
The biggest difference between surveys and interviews is that there’s no real-time interaction, so you cannot react to someone’s response right away and ask follow-up questions. As a result it’s hard to dig deeper and find out the “why” behind someone’s answers. Next to that you’re less likely to stumble upon entirely new insights and get those “aha!” moments.
However, whereas you need to take some time for every interview and it’s thus not scaleable, you can easily make one survey and send it out to many people! So surveys are great for gathering qualitative input from a large group of people, and (in)validating assumptions you have about your target group. In that sense it’s a great follow-up to a bunch of interviews! (First discover, then validate)
Yet I’ve seen many many people sending out surveys instead of doing interviews, because they think interviewing will take too much time, or are simply a bit scared to do so (I know, it’s always a bit nerve wracking). However, you’ll quickly find out that 1. Surveys also take a lot of time to set up, so are only really cost-effective at scale, and 2. You’ll probably get a lot of answers you cannot place because you don’t know the context.
So the take-away: Use interviews for exploration and getting new insights, and surveys for validating and checking whether something is true for a larger group of people. That’ll help you to get the most out of both methods :)
Just like when doing interviews, “would you..?” questions won’t give you very reliable answers when being asked in surveys either. People are notoriously bad at predicting their future behaviour, which is why the answers to “would you” questions should be taken with a big grain of salt.
Instead of asking people to predict their future behaviour, ask them about their past behaviour instead. Their actions, what products they are currently using, how often they used something over the last week. As that’s a much better predictor of future behaviour.
Ever answered one simple question, after which a form with 20 more questions popped up? And thought to yourself “goddammit I don’t have time for this right now”, and clicked it away? (To never get back to it.) That’s what happens when you present a long survey to your respondents without being up-front about it; you’ll lose goodwill, and people are unlikely to fill it out.
A raffle is not the best idea either: people might fill out your survey, but do so in a “lazy” way, i.e. just clicking on random answers to get to the end. The worst thing about this is that you’ll end up with bogus answers that’ll muddy your data.
In general it’s good practice to stick to 10 questions max if you want to increase the chances of your survey being filled out.
However, longer surveys are an excellent way of testing for interest and commitment. Because if respondents are willing to endure a 30-min survey, they must be interested in the topic! This is called a high hurdle test. In this case just make sure to make the purpose of the survey clear up-front as well as how long it will take to fill out, and don’t provide any incentives that are notrelated to your service or product (so an invite for the beta test it an excellent incentive, an Amazon gift card not so much).
Even though you’re limited in asking follow-up questions in a survey vs an interview, it’s still a good idea to include some open-ended questions! It’ll allow someone filling out your survey to leave any additional information and provide more context. This in turn can help you understand their answers better, and gain new insights. Afterwards you can for example follow up with an interview to find out even more about it.
We for example experienced this when sending out a survey for our tool Heatcheck, and getting a respondent who indicated he identified with being a maker looking to launch his new product, a founder with an already existing product, a corporate innovator, and a product manager within a big organisation! Turned out he was a consultant who had helped out people in all these situations.
Another very nifty thing you can do with input to open questions, is use them as marketing material! These are the exact words your target group wrote down, and it’s great for understanding what terms they use to describe the problem you’re trying to address, and in what language you can reach them best.
Which is crucial because it’s easy to overlook things when you’ve been working on it for hours / days on end. Plus, as you already know a lot about the topic, you’ll probably have a different frame of mind from the average person who stumbles upon your survey.
In this case it really helps to get a fresh pair of eyes to look at your survey and give feedback, and afterwards send it out to a small group of people to test, before pouring your euros into online campaigns and ads to spread it to the masses. This so you can make sure your questions are clear and unambiguous, and respondents know exactly what you’re asking for.
Else you run the risk of getting widely varying answers you cannot draw any conclusions from, or worse, information that’s simply wrong and puts you on the wrong track (or takes you off the right one!).
And really, testing can be done very quickly and doesn’t have to be hard. Just quickly show it to some people and ask them to fill it out while thinking out loud. Or send it out via your network to see what initial answers you get, then follow up to find out why they answered the way they did.
In which case you’re missing out on a big opportunity! Namely, to find out more about why respondents answered the way they answered, but *most of all *to find out how they came across your survey, and why they took the effort to complete it!
This can give you great clues to what problems people experience that your product or service could address, under what circumstances they start looking for a solution (i.e. what are their pains?), and where they do so (great for marketing and picking your channels!).
Apart from that surveys are a great way of setting up an email list of potential customers to follow up with later on & do follow-up testing with. If you can, always go for increased commitment!
Lucky you, at the moment we’re building a tool for just that! Heatcheck helps you create quality surveys for testing your business idea in the market. Just answer some questions around your project, and we will get you a survey in no-time!
Interested? Go here to sign up and get your first survey for free!
