How to create an effective questionnaire

If designed well, questionnaires can be useful tools to complement interviews and facilitated group discussions to gather information from stakeholders to inform your strategy or improve your customer experience.

The key to successful questionnaires is being clear about what you want to achieve, and making sure the questions you ask will give you the insights you need. Sounds simple but it’s not that easy to do in practice.

Here is my five-step approach in a nutshell:

• Step 1: Write down your questionnaire objectives in detail
• Step 2: Think about the motivation of your target participants
• Step 3: Tailor the implementation of the questionnaire to your objectives
• Step 4: Think about flow
• Step 5: Close the circle: imagine you have the questionnaire data

Step 1: Write down your questionnaire objectives in detail

Dig deep and question what outcome(s) you exactly want to achieve – not just the information you wish to collect but what it will enable you to do. Capture each distinct desired outcome as a separate objective. Be very structured in analysing your objectives. Consider splitting an objective into distinct sub-objectives if needed.
Avoid having too many objectives as this will dilute the focus and effectiveness of your questionnaire. Prioritise your objectives into essential, useful and nice-to-have.

Step 2: Think about the motivation of your target participants

Consider the interests of the participants you will be inviting to take part in your questionnaire. Why would they want to fill out your questionnaire? What concerns or interests may participants want to share with you? What do they care about? In other words, what is in it for them – beyond the chance of winning a voucher? As a questionnaire respondent I often think that I’m being asked all the “wrong” questions and that what really matters to me about the topic isn’t part of the questionnaire. Consult with a few key participants or people who are representative of your target audience to check your assumptions.

Step 3: Tailor the implementation of the questionnaire to your objectives

Take each objective in turn, and consider what the best question format is to achieve the objective.

Two main considerations:

• The type of information you need: Are the possible answer(s) limited to a fixed set of options (such as the days of the week), do you need participants to give you a rating (e.g. a score out of 5) or would you benefit most from participants giving you insights in their own words?
• The experience of the participants: What format will make it easy for participants to accurately give your their views? Both "easy" and "accurate" are important.

I’m not mentioning ease of analysing the data. In my view the analysis needs to follow from the type of data you need to collect, which in turn follows from your objective. Data from a fixed set of options (radio button or check box) is a lot easier to analyse than free text. However, if you are looking for insights then giving respondents pre-set options carries the risk of biasing the results – unless you already have reliable insights from in-depth qualitative methods (such as interviews and facilitated group discussions) and your aim is to refine/validate these with a larger group of people.
Consider the wording of each question carefully – make sure your question matches the objective and that the wording doesn’t bias respondents’ answers.

Step 4: Think about flow

Consider what sequence of questions will flow best – both to achieve your objectives and to give participants a smooth experience. Questions that seem to be in random order will jar and respondents may abort the questionnaire. The sequence of questions needs to follow a certain logic – there is no one right order but the questions need to flow. For example, questions about one objective could be grouped. You may want to start with more general questions and narrow down to specific aspects.
Also keep in mind that the order in which you ask questions can influence the answers you get! This is because one question may prime respondents’ minds and influence their answer to a later question.
Revisit your prioritised list of objectives – should you discard the nice-to-have objectives to keep the questionnaire concise and focused?
Research has shown that if you ask for respondents’ demographic details at the start of the questionnaire, they are more likely to abandon the questionnaire so – if needed – ask for this data at the end (and make these questions optional, of course).

Step 5: Close the circle: imagine you have the questionnaire data

Imagine that your questionnaire has been concluded and you have the data. What now?

Consider, from your own perspective:

• Do you have the data/information you need to meet your objectives?
• Will all the data you have collected be relevant and useful? Should you scrap questions?
• How will you analyse the data? How will you draw relevant conclusions?

Consider, from the participants’ perspective:

• Did they have a positive experience? Are they likely to continue till the end?
• Did they have the opportunity to share their concerns or interests?

If necessary, revisit Steps 3 and 4 and adjust the question format and wording and the flow of your questionnaire.
You are now ready to start piloting your questionnaire with a small sample of well-chosen test participants.

Feel free to drop me a line to share your thoughts — catherine [at] strategic-consulting.scot