8 things you need to know before you start writing a questionnaire

Sep 7, 2023 | General

Close up photo of allium flowers, purple star shaped flower with a small green centre

Questionnaires are a commonly used method.  Very commonly used. They are often perceived to be a relatively simple way to understand what a target human population think, know or feel about a given topic. 

And they are a great tool.  But they are a lot more complex for a researcher to use than first appears. 

Many people have been caught in the quagmire of questionnaires, tripped up by the hidden complexity of this method.

If you’re thinking about using a questionnaire to gather data, here’s a quick refresher on what questionnaires are, and my top tips for what you need to know before you start drafting those questions. 

Refresher: what are questionnaires?

They are a set of questions which are answered by the respondent independently of the researcher/data collector.  The same questions are asked to all respondents.

Questionnaires are generally: short, take only a few minutes to complete, and collect predominantly quantitative (numerical) data.  This makes them a great way to gather responses from a large number of people – from hundreds up to tens of thousands of respondents.  This scale of data collection allows researchers to understand large scale patterns with the population.

But – and it’s a big but – questionnaires come in many formats.  They can be long, gather qualitative (text-based data), have a small sample size and be used to explore a topic in much more depth.

We’re already exposing some of the challenges of questionnaires in the diversity of formats they come in!

To help you navigate those challenges, here are eight questions I recommend that you are able to answer before you start drafting a questionnaire…

1. What is your research question?

Your research question is the overarching question you are trying to answer.  Be REALLY clear on this as early as possible.  It may evolve during the project, but your research question gives you the parameters to work within as you develop your data collection.  If you get stuck at any point, coming back to your research question is a great way to get moving again.  If you can’t get a research question defined, at least have a research aim.  Asking people questions is interesting – people are interesting.  If you don’t have a research question or aim you can fall into the trap of asking your respondents lots and lots of interesting questions.  This takes up a lot of time, gives you way more data than you can analyse, and can take you in too many directions at once.

2. What data do you need? 

The questionnaire needs to gather the data you need to answer your research question.  So, if your research question is “How does engagement with the coast affect people’s connection to nature?” you need data on the respondent’s engagement with the coast, and data on their connection to nature.  You can then break down the data need into potential measures – e.g. engagement could be measured by questions about frequency of visits to the coast, duration of visits to the coast, types of activities done when at the coast – or a combination of these.  Defining these data needs helps you to identify potential questions, which can be helpful if you’re not sure where to start.

3. How are you going to analyse the data? 

Yes, I’m asking you to start thinking about data analysis before you’ve drafted a single questionnaire question.  Different question types require different analysis.  What analysis skills do you – or your research team – have?  Your available skills may influence the types of questions you’re able to analyse.  For example, there’s no point asking respondents to rank a list of items if you don’t have the statistics skills to analyse ranked data.  Consider whether you need to ask closed or open questions?  Closed questions gather numerical (quantitative) data, open questions gather text responses (qualitative data).  It’s ok to have a combination of both – but the data will need to be analysed.  If you haven’t analysed social science data before, find someone who can support you to do it.

4. What other work has been done on your topic? 

I’m hoping you’ve looked at the literature before you got to question 1 on this list!  But revisiting the literature when you’re starting to design your questionnaire can be useful.  Understanding what methods other researchers in your topic have used, questions they’ve asked or particular metrics that may be relevant can be very useful. 

You might discover that the topic you’re interested in is a concept which has years or decades of research behind it.  This is likely to be the case if you’re looking at conservation behaviours, connection to nature, food security, empowerment, social values…I could go on.  Many of the topics people talk to me about researching have frameworks, insights and are already being explored in some form or another.  So always make sure to check what has been done before and how you can build on that.

5. Who is your target population? 

This question is about being clear on who you need to gather data from.  It might be farmers, cyclists, visitors to a particular place, residents who have lived in an area more than 10 years etc.  And you may have more than one target population.  Use parameters such as place of residence, socio-demographic variables, employment in a particular industry, role in the community etc to define them.  I always recommend caution around having the “general public” as your target population.  It usually requires a reasonable budget to gather data from a sample size which is big enough to be truly representative of the general public.  If you have a small budget for data collection a more tightly defined target population may be more achievable.

Knowing who your target population is provides vital information for choosing your method, sampling design and target response size, questionnaire distribution, and generally delivering a successful research project.

6. How will you distribute the questionnaire?  

Your questionnaire needs to get in front of people who are from your target population.  If you are using a paper questionnaire, you might be using a postal questionnaire, sent in the mail with a return envelope.  Or delivering it house to house in a selected residential area and returning to collect it a certain number of days later.  If you are using an online questionnaire there may be online groups where you can invite respondents, email distribution lists or social media platforms which are used by your target population.  Remember to check the rules for any groups you’re distributing in online. 

Have a strategy for getting your questionnaire in front of people.  And once the questionnaire is live, track whether you’re getting responses.  Be ready to amend your sampling approach if you aren’t getting the response rates you need.

7. Are you sure a questionnaire is the right method? 

If you and I were talking about your research, I’m likely to ask you why you’ve chosen to use a questionnaire.  Answers like “umm…I’m not sure what other method I could use” or “because we used a questionnaire on another project so I’m using the same one here” are going to sound an alarm bell for me!  I get it.  Questionnaires look simple and it can feel like a relatively easy way to gather data.  But they are deceptive – questionnaire development is much more complex than the questionnaire which ends up in front of your respondent.  (Perhaps that’s why you’re reading this blog?!)

The ”right” method for a project needs to be a method which will: 1) collect the data you need to answer your research question; 2) enable your target population to engage in the data collection; and 3) matches the skills and capacity of the research team to design, deliver, analyse and interpret.  Reflect on your reasons for choosing a questionnaire, and whether it is the best method for the data, target population and research team.  If it’s not the right method, don’t be afraid to change.

8. Is your research ethically sound? 

Last, but certainly not least, your research needs to comply with human ethics principles and data protection regulations.  This may sound rather dull, but these processes support you to do the best quality research by ensuring the respondent’s needs are considered.  If you have never heard of human ethics or aren’t sure what to do about data protection, then search out some more information.  If you’re part of a University or research institute there should be a Human Ethics for Research process (maybe a committee or contact person).  If there isn’t, find a partner organisation or someone in your network who can advise you. Data protection rules vary by country so find out what is required of you and your organisation.  Get started on this as early as you can.

To wrap up, two pieces of advice…

Firstly, if you don’t know the answers to the questions I’ve asked here, pause the questionnaire writing and do the research or find people to talk to who can support you.  If you are new to questionnaires, or have struggled with questionnaires in the past, put the word out to get some help.  And the sooner you ask the better your research will be. 

Secondly, once you’ve got the answers to these questions, it’s time to start drafting – not writing! When you start developing the actual questions for the questionnaire, know that you will do multiple drafts.  No one ever sat down to write a questionnaire, wrote it, and distributed Questionnaire v1.0.  This is a process of writing and honing until you get to the questions which collect the data you need to answer your research question. 

Questionnaires are a great method when used well.  I hope this blog helps you navigate your research design and avoid some of the pitfalls of this useful method.

And if you want to know more about social science and the diverse methods used to understand people, check out our Essentials of Conservation Social Science course.

0 Comments

Submit a Comment

Your email address will not be published. Required fields are marked *

Related
Posts:

Reflections on starting a business

Reflections on starting a business

Three and a half years ago I stepped away from employment in a large conservation organisation to launch my own business - Human Nature.  I’ve recently been asked a few questions about this shift from employee to independent.  It’s been an exhilarating few years. ...

read more
Coastal Futures 2023

Coastal Futures 2023

A few weeks ago Human Nature was one of the sponsors of Coastal Futures 2023.  This conference of 350 people in London and several hundred more online brought together coastal and marine managers, researchers and practitioners to consider the next decade for the...

read more