Write more effective survey questions
14 tips to help you improve your survey question writing
Survey items have two separate parts: the question and the answer. The distinction is important. A good question not only asks for information clearly, but elicits useful responses.
9 rules for writing good questions
Naturally, no question is "good" in all situations, but there are some general rules to follow. Using these rules and examples will help you write useful questions.
1. Remember your survey's purpose
All other rules and guidelines are based on this one. There was a reason you decided to spend your time and money to do your survey, and you should ensure that every question you ask supports that reason. If you start to get lost while writing your questions, refer back to this rule.
2. If in doubt, throw it out
This is another way of stating the first rule, but it is important enough to repeat. A question should never be included in a survey because you can't think of a good reason to discard it. If you cannot come up with a concrete research benefit that will result from the question, don't use it.
3. Keep your questions simple
Compound sentences force respondents to keep a lot of information in their heads, and are likely to produce unpredictable results. Example: "Imagine a situation where the production supervisor is away from the line, a series of defective parts is being manufactured, and you just heard that a new client requires ten thousand of these parts in order to make their production schedule. How empowered do you feel by your organization to stop the line and make the repairs to the manufacturing equipment?" This question is too complex for a clear, usable answer. Try breaking it down into component parts.
4. Stay focused - avoid vague issues
If you ask "When did you last see a movie?" you might get answers that refer to the last time your respondent rented a video, when you are really interested in the last time the respondent went out to a movie theater.
Consider too, "Please rate your satisfaction with the service you have received from this company." This is a fine general question, but will not likely lead to any specific action steps. Particular elements of service must be probed if responses are to result in specific recommendations.
5. If a question can be misinterpreted, it will be"What time do you normally eat dinner?" will be answered differently by people living in different regions; "dinner" can refer to either the midday or the evening meal. Be clear, concise, always beware of imprecise language and avoid double negatives.
6. Include only one topic per question (avoid "double-barreled" questions)How would you interpret the responses to "Please rate your satisfaction with the amount and kind of care you received while in the hospital." or, a question asking about speed and accuracy? If you want to be able to come up with specific recommended actions, you need specific questions.
7. Avoid leading questionsIt is easy, and incorrect, to write a question that the respondent believes has a "right" answer. "Most doctors believe that exercise is good for you. Do you agree?" is an example of a leading question. Even the most well-meaning researcher can slant results by including extraneous information in a question. Leading questions can be used to prejudice results.
8. Consider alternate ways to ask sensitive questions
Some questions are obviously sensitive. Income, drug or alcohol consumption and sexual habits are clear examples of topics that must be asked about carefully. The question: "Did you vote in the last election?" has an element of sensitivity in it as well. Respondents might be unwilling to admit that they did not vote, because of civic pride or embarrassment. To avoid respondent alienation, it can be useful to mitigate the cost of answering "No" by including a way out. For example: "There are many reasons why people don't get a chance to vote. Sometimes they have an emergency, or are ill, or simply can't get to the polls. Thinking about the last election, do you happen to remember if you voted?"
Also, people are less likely to lie about their age in face-to-face interviews if they are asked what year they were born, rather than how old they are.
9. Make sure the respondent has enough information
Asking respondents "How effective has this company's new distribution program been?" may not be as effective as "Recently, we implemented a new, centralized distribution system. Did you know this?" Followed by "Have you seen any positive benefits resulting from this change?" It can be beneficial to break down questions that require background information into two parts: a screening item describing the situation which asks if the respondent knows about it, and a follow-up question addressing attitudes the respondent has about the topic.
5 rules for obtaining usable answers
Useful answers are just as important as good questions. Here are some rules:
1. Response options need to be mutually exclusive and exhaustiveThis is the most important rule to follow when providing response options. If response options are not mutually exclusive, the respondent will have more than one legitimate place for their answer. The response choices, "1 to 2," "2 to 3" and "More than 3" pose a problem for someone whose answer is "2."
You must also ensure that the response options you provide cover every possibility. Asking "Which of the following beverages did you drink at least once during the past seven days?" and providing a list of coffee, soda and tea might be sufficient if you were doing a study on the consumption of caffeinated drinks. But, they would not work if you wanted to know about broader consumption habits. If you are unable to provide a complete list of options, at least provide an "Other" choice. If the list of choices is too long, an open ended-question might be a better option.
2. Keep open-ended questions to a minimumWhile open-ended (or verbatim) questions are a valuable tool, they should not be over-used. Not only can they result in respondent fatigue, but they pose problems in terms of coding and analysis.
3. People interpret things differently, particularly when it comes to timeTrouble-spots include responses such as "Always," "Sometimes" and "Never." You must build in a temporal frame of reference to ensure that all respondents are answering in the same way. As in this example from an interviewer-administered questionnaire, "I am going to read a list of publications. For each one, please tell me whether you read it regularly. By regularly I mean, at least three out of every four issues."
4. Consider a "Don't Know" responseIt is useful to allow people to say they simply do not have an opinion about a topic. However, some investigators worry that people will opt for that choice, reducing the ability to analyze responses. Evidence shows that this fear is largely unfounded. The goal of your research should help you decide if a "Don't Know" option would be wise. For example, if you only want information from those with an informed opinion or higher interest, offer a "Don't Know" choice.
5. Provide a meaningful scale
The end points of response scales must be anchored with meaningful labels. For example, "Please rate your satisfaction with customer service. Let's use a scale where 1 means 'Very Satisfied' and 5 means 'Very Dissatisfied.'" You could also give each point on the scale a label.
The number of scale points (3, 5 or 7) can have little effect on the conclusions you draw later. Choosing how many points, then, is often a matter of taste. There are three things to remember when constructing a response scale. First, an odd number of points provides a middle alternative. This is a good way to provide respondents with moderate opinions a way out (similar to the "Don't Know," choice above). Secondly, if measuring extreme opinions is critical, use a scale with a greater number of points. Finally, you generally gain nothing by having a scale with more than 7 points and will probably find that you will collapse larger scales when it comes time to analyze the data.
The price of poorly written questionsWell-written questions are critical. Participants must stay interested. If your respondents start to feel alienated by threatening, emotional or difficult questions, response rates are likely to go down and response bias will probably go up.
Also, respondents can get frustrated if your questions do not provide answer choices that match their opinions or experiences. The quality of your collected data will suffer; your analyses will be less meaningful; and the whole research process may prove useless or harmful. So think carefully about the questions you write, look at reputable examples of questions, and refer to the rules above. If you follow these guidelines, you'll do fine.
What is this a picture of?
