Advice on Survey Design

Surveys are very important and helpful in collecting useful information from a wide range of people. The proper planning and design of surveys is critical, as poorly planned or designed surveys can affect the quality of the information collected and negatively impact future response rates.   

This is just a small set of advice. For more information, Qualtrics has various articles you can read as a starting point. Check out "11 Tips For Building Effective Surveys" and "Survey Design Your Respondents Will Love."

1. Do Your Background Research


If you are seeking to measure an attitude, behaviour or concept there may be previous research done on that topic. Review the literature carefully and you might find existing questions, scales and approaches that can be used in your survey, and the reliability and validity of these may already be established.   

2. Embrace Backwards Design


Every survey should have a purpose and clear goals and outcomes. Working backwards through your analysis plan, population sample plan, survey instrument development, and implementation plan will help you focus on how you will use the information gathered from your survey and keep the survey instrument and analysis better aligned, lowering the chances of having a survey result in “nothing”.

3. Do the Groundwork


Conducting a small focus group with your target population is a good way to determine important issues, what kinds of questions to ask and how best to phrase them, and what to include in question response categories. Doing so will help build validity for your survey and anticipate and address issues before you build and launch your survey.

4. Keep Your Survey Short


No one likes an overly long survey, so keep only the items you absolutely must ask. Having a focused survey reduces the time that people will spend on it, making it more likely that they will complete it and provide the information you need.

5. Question Order Matters


Keeping in mind that any question you ask can influence later responses, your survey should be presented like an interview. Start with an introduction and the easy questions, lead into the more difficult ones, and keep the administrative questions (i.e. demographics) to the end.

6. Open & Close-Ended Questions


Open-ended questions let participants freely describe and detail their answers and present a wealth of information but are often time-consuming and difficult to properly analyze. Consider well-constructed close-ended questions that are easier to analyze and use related questions to probe responses to larger multi-faceted issues.