INFORMATION LINE 1300 85 85 84
Surveys may be one of the first things we consider when we think about evaluation, but they aren’t always as easy to design as they might seem. Below are some examples of the things that you need to consider when compiling a survey.
A well-written survey carefully considers:
There are a number of important considerations when writing survey questions, as well as some common traps when it comes to wording your questions clearly. We go through these below, and include a relevant example question where appropriate.
Often, we use pre-and-post survey to get feedback on a program from our target audience. It’s easy to confuse a participant’s enjoyment of a program or activity with the program or activity meeting its objectives.
For example, the Victorian Police, as part of the evaluation of their SSMART ASSK program with young people, stopped using beer goggles in part because student feedback indicated many found them to be “one of the best things about the program” with an overwhelming emphasis on how “fun” they were. This supported previous concerns that the “fun” nature of the activity was undermining the knowledge it was intended to provide, and that it risked glamorising the effects of alcohol instead. After conducting a review of the evidence and finding it was not best practice, the use of beer goggles was discontinued.
It is important to be mindful of ethical considerations is a critical part of engaging with any target audience, and there are some specific issues to keep in mind.
Why do we need to consider ethics? It is essential that the way we collect our information does not harm to those we are collecting it from, as this will undermine our overall aims.
What do we mean by harm? Harm can be experienced in many different ways by people, in relation to AOD is it important to make sure that we don’t make people feel isolated, stigmatised or judged by the questions that we are asking. When working with vulnerable groups this is even more important. When working with young people we need to make sure that our surveys are age appropriate. Some ways to ensure our work is ethical is to be mindful of the following:
It can be tempting to ask our respondents everything we’ve ever wanted to know about their demographic. However, care must be taken to restrict our data collection to that which will be directly useful to our program in respect of their privacy and time. Additionally, people are more likely to complete shorter surveys.
Working with some populations, like under 18s, require both ethics committee clearance and the consent of their parent/guardian. Working with other ‘vulnerable populations’ may require ethics clearance.
Particularly when it comes to alcohol and other drugs, care needs to be taken not to reinforce exaggerated perceptions about how many people use or are dependent on drugs, the messaging that all drug use is dependent use, the idea that illicit drugs cause more harm than alcohol, etc.
This requires careful wording, and age-appropriate, context-sensitive questions.
These are unintended consequences of certain types of questions or question wordings.
These are common pitfalls when writing questions that can make your survey hard to understand, take longer to complete, and may even compromise your data.