Design the Survey

Maximise survey responses, drive positive change, and transform engagement through effective data collection.

How do we design a survey?

The design of your survey is an integral part of a successful stakeholder feedback exchange. This is the tool you’ve chosen to figure out the key elements related to your goal, so it must be purpose-driven, well-rounded, and provide you with the types of data you are looking for.

Choose the survey type

Is the topic something you are exploring for the first time or is this going to be a continuation of some work that you’ve done previously? This is where you commit to either a new or repeat survey. Once you know this, you can start working on other considerations for the survey you wish to create.

Identify survey respondents

Who needs to fill out the survey to give you the answers and clarity you need? In schools and trusts, usually, these are pupils, staff or parents. It’s important to know who will fill out your survey to ensure you use the appropriate invitation, questions, language, technology, and timing to gain their insight.

Identifying the target survey respondents will make sure that the correct questions go to the correct people who will provide us with the most useful data about the topic we’re interested in.

Survey topic

KEY STAKEHOLDERS (receiving information about):

Staff members

RESPONDENTS (giving information)

Staff members or other possible respondents:

  • Pupils (to ask about behaviour which may be affecting staff wellbeing)
  • Parents (to ask for views on levels of trust and respect for school staff)
Explore templates →

Within each survey, we recommend you include a few demographic questions, which will help you with analysis later in the process. To deepen the survey analysis and your understanding of the stakeholder groups, it is valuable to also add additional demographic questions on protected characteristics like gender, ethnic group, disability, sexual orientation, etc., which we currently ask to students and staff.

Decide on the anonymity of the survey

When any respondent is requested to provide their voice and reflection, one of their main concerns is “Who will see this and in what form?” While some individuals will be comfortable answering honestly, regardless of who sees their answers, others will be concerned about the protection of their attitudes and feelings for fear of a negative impact or reaction to them. It is therefore crucial to guarantee the anonymity of individual survey responses.

We, at Edurio, take this commitment exceptionally seriously and do our part to limit the identification of individual voices. We strongly recommend you make your surveys truly anonymous and avoid the complex dynamics that arise with identified surveys.

Anonymity provides:

The respondent with a sense of ease and contributes to greater honesty in their responses, and opens to a greater variety of voices in your organisation

Less biased responses due to personal relationships or power dynamics

Reduced likelihood of respondents giving socially desirable responses (telling what they think you want to hear)

Higher response rates mean that your results will have a better stakeholder body representation

How do we write the content of a survey?

Designing a good survey question primarily means
two things:

  • The question is valuable to you
  • The respondent can give a meaningful answer

#1
Generate ideas

If you are creating your own questions for the survey (rather than customising or re-using questions from a previously built survey), we suggest gathering your team together and throwing out all the ideas that come to mind around the following prompts:

  • What are the different areas to explore to get a clear and well-rounded picture?
  • What are the most direct questions we can ask about this topic to get clear answers?
  • What questions might help us understand the context?
  • What questions might give us clear guidance about the best direction/solution for a situation?

In this step, don’t worry about the language or the type of the question, just ask it! This step is about getting all the ideas out so you have a starting point to design your survey.

Warning

Stick to only asking questions that will be valuable to you – ask about things that you don’t have answers to, or that help you validate your hypotheses, and that you can use to inform your actions post-survey. The questions you ask should be geared towards helping you achieve the goals you set out during planning. It can be tempting to include “interesting to find out” questions, but remember that every question you add is an additional bit of work for your respondent, as well as yourself when you’ll be exploring the data after the survey.

#2
Focus your survey

Shift from the idea-generating stage to giving your survey structure and shape by:

  • Placing similar questions together by theme;
  • Removing duplicate questions so you can see the full list of questions;
  • Ensuring you have a well-rounded question set for each of the themes you’re interested in;
  • Naming the different themes/buckets of questions;
  • Going back to your goal and discussing which themes are essential to get the information you need for this survey to be successful;
  • Deciding which themes to keep and which to skip for this survey. Remember, more is not better when it comes to a survey!

Once you’ve identified your themes and have drafted questions, it’s time to dive into cleaning up the questions and deciding the best way to invite participants to answer them.

#3
Question types

Single answer questions

Respondents choose one response of the choices you have provided.

Example

Multiple answer questions

Respondents select all of the responses that apply to them.

Example

Open questions

Respondents write the answer to the question in their own words.

Example

#4
Key factors of question design

When writing your questions, focus on asking about the observations and feelings
of your respondents, taking into account:

Pro tips

#4
Think about the language

Be specific and intentional in the language you use and focus on the respondents’ point of view. As you’re writing the questions, ask yourself:

All of these considerations will help you create questions that take less effort to answer, which can greatly improve the overall survey-filling experience of the respondent and, in turn, improve the quality and quantity of the data you receive.

Choose appropriate response options

As important as the quality of your question text is to get the key information from respondents, the type of response format that you choose to pair it with is equally vital. There are different types of response formats and each of them gives us various types of information.

At Edurio, we mostly create surveys using various types of multiple choice questions with a combination of open and demographic questions (demographics are explained in detail above).

 

Multiple choice questions

Multiple choice, also known as closed-ended questions, provide a list of responses to choose from and will give you quantitative information about the topic. Multiple choice questions can ask for a single or multiple responses to be selected out of the options presented.

Why 5 response options is a good format

At Edurio we mostly use 5 response options for questions. When compared to a 3-point scale, the 5-point provides more granular information, which allows for more in-depth analysis where necessary. We often find the 7 response option to offer information that is too granular, requiring more time from the respondent than we believe is needed. As an odd-numbered scale, it also provides a natural midpoint between the most positive and the least positive answer options.

We understand that you want to get concrete answers to your questions. But we need to remember that respondents are people with their own experiences and feelings, therefore including “middle of the road” options is a part of life.

On rating scales, make sure the respondents have the option to select a neutral response option, for example, “Sometimes”, “Neither easy nor difficult” or “Moderately confident”, as we believe that being neutral about something is a valid part of the human experience and it’s important to have a chance to be reflected in the data.

Useful response categories

If you want to know how often something occurs, ask for the exact number of times or ask “How often…” and choose from the following categories:

If you want to find out the degree of people’s feelings, attitudes, or beliefs, try to ask the question directly: “How confident?”, “How comfortable?”, “How well?”, “How easy?” etc., and use the following response scale:

However, if you want to formulate the question with “To what extent…” then use the following set of responses:

If you want to find out people’s intentions or check if something has taken place, you can start a question with “Do you…“, “Have you…” or “Will you…” and ask them to choose from the following categories:

It’s essential that your question and response options are mutually exclusive and collectively exhaustive (also known as MECE), meaning that these groups do not overlap and that they cover all possible options in the given context.

For example, if you’re asking pupils about people they talk to about their troubles, make sure you create clear categories that don’t overlap (like “friends” and “ online friends”) and that reflect groups with whom they are most likely to interact. This means also accounting for the fact that you might not have thought of everybody and including an option “Someone else (please comment)” to give them an opportunity to share it.

To make the options truly MECE, take into account that maybe they don’t share their troubles much outwardly and add an option “Nobody”. Give respondents the option to provide a response like “Other”, “Don’t know”, or “Not applicable” where applicable or to not reply at all. For instance, for the question, “When you feel sad or worried, who do you talk to?”…

Instead of this

Friends
Parents
Family
Teachers
Friends online

Use this

Classmates
Friends outside of school
Online friends
My parents
My siblings

Teachers
Someone else (please comment)
Nobody

How do we structure the survey?

Once we have our questions and answer formats sorted out, it’s time to think about making the surveying experience as pleasant as possible for the respondents. The respondent burden is the effort that it will take for respondents to complete a survey from start to finish. Lessening the burden can improve the chances of respondents giving quality feedback and increase their satisfaction about having taken part in the survey.

Some of the elements contributing to respondent burden will be beyond your control, but being clear about the contents of the survey, why this feedback is important to you, and how you plan to use it, will help them be more committed to participating fully.

#1
Consider the length

The length of the survey can have a direct impact on the response and completion rates. The longer the survey, the less likely respondents will feel inclined to complete it or to start it altogether. In our experience, 60 questions or less is an optimal length for a diagnostic type of survey as it allows adults to complete in about 15 minutes, without it having a negative impact on the quality of the data received.

If your respondents are children, young people or people with special educational needs, then you should take this into account and commit to a smaller number of questions in the survey and, in addition, consider other methods of collecting feedback. In our experience the optimal number of questions for these groups are starting from 20 to 55.

As funny as it may sound, the most impactful tool at your disposal is the delete button! Every question you ask that you don’t take action on is a waste to you and your respondents. So, as much as you can, review and edit down where possible.

#1
Be mindful of the order of questions

Much can be achieved by putting thought into how the survey questions are ordered. This can go a long way towards a surveying experience that is logical and smooth for the respondent.

#1
Clarify and guide where needed

There are a few additional tools at your disposal that can help make the survey experience more pleasant for those participating. Please note that the ability to include these elements will depend on the survey tool you choose to work with.

How do we test a survey?

To make sure you’ve designed a survey that will help serve your organisation’s needs and achieve your goals, it should be tested with your intended respondent group. This enables you to figure out what to revise and amend prior to launching the survey.

First, try thinking about what you would do with the results – would you know how to interpret the responses? Then, pretend to be a respondent for a while and try to “break” your survey by imagining a wide variety of opinions that would need to be captured by these questions – can your survey accommodate all situations? Finally, pilot the survey to test it with actual respondents.

Key terminology

Change agent: Someone who is taking the time to study and guide the successful process.

SLT/Exec Representative: Senior Leadership Team, Executive Team. Depending on the structure of your school or multi-academy or single-academy trust, the leadership team structure may differ.

Stakeholder Representative: Staff, parent or pupil representing the group for your project

Materials & templates

One informational summary resource accompanies this chapter of the hub. You can share and adapt the materials by providing a link to the original documents and indicating if changes were made. You may do so in any reasonable manner but not in any way that suggests that Edurio endorses you or your use.

COMING SOON

Survey design checklist

All additional resources & templates are available for free: The page is password-protected; you may retrieve the password by completing the form above. After form completion, an email containing the password will be delivered to your inbox.

View materials

Share with friends and co-workers!