Questionnaires refer to a series of questions asked to individuals to obtain statistically useful information about a given topic. When properly constructed and responsibly administered, questionnaires can provide valuable data about any given subject.
Questionnaires are frequently used in quantitative marketing research and social research. They are a valuable method of collecting a wide range of information from a large number of individuals, often referred to as respondents.
What is often referred to as adequate questionnaire construction is critical to the success of a survey. Inappropriate questions, incorrect ordering of questions, incorrect scaling, or bad questionnaire format can make the survey valueless, as it may not accurately reflect the views and opinions of the participants.
Different methods can be useful for checking a questionnaire and making sure it is accurately capturing the intended information.
Initial advice may include
consulting subject-matter experts
using questionnaire construction guidelines to inform drafts, such as the Tailored Design Method, or those produced by National Statistical Organisations.
Empirical tests also provide insight into the quality of the questionnaire. This can be done by:
conducting cognitive interviewing. By asking a sample of potential-respondents about their interpretation of the questions and use of the questionnaire, a researcher can
carrying out a small pretest of the questionnaire, using a small subset of target respondents. Results can inform a researcher of errors such as missing questions, or logical and procedural errors.
estimating the measurement quality of the questions. This can be done for instance using test-retest (), quasi-simplex (), or mutlitrait-multimethod models ()
predicting the measurement quality of the question. This can be done using the software Survey Quality Predictor (SQP ).
Know how (and whether) you will use the results of your research before you start. If, for example, the results won't influence your decision or you can't afford to implement the findings or the cost of the research outweighs its usefulness, then save your time and money; don't bother doing the research.
The research objectives and frame of reference should be defined beforehand, including the questionnaire's context of time, budget, manpower, intrusion and privacy.
How (randomly or not) and from where (your sampling frame) you select the respondents will determine whether you will be able to generalize your findings to the larger population.
The nature of the expected responses should be defined and retained for interpretation of the responses, be it preferences (of products or services), facts, beliefs, feelings, descriptions of past behavior, or standards of action.
Unneeded questions are an expense to the researcher and an unwelcome imposition on the respondents. All questions should contribute to the objective(s) of the research.
If you "research backwards" and determine what you want to say in the report (i.e., Package A is more/less preferred by X% of the sample vs. Package B, and y% compared to Package C) then even though you don't know the exact answers yet, you will be certain to ask all the questions you need – and only the ones you need – in such a way (metrics) to write your report.
The topics should fit the respondents’ frame of reference. Their background may affect their interpretation of the questions. Respondents should have enough information or expertise to answer the questions truthfully.
The type of scale, index, or typology to be used shall be determined.
The level of measurement you use will determine what you can do with and conclude from the data. If the response option is yes/no then you will only know how many or what percent of your sample answered yes/no. You cannot, however, conclude what the average respondent answered.
The types of questions (closed, multiple-choice, open) should fit the statistical data analysis techniques available and your goals.
Questions and prepared responses to choose from should be neutral as to intended outcome. A biased question or questionnaire encourages respondents to answer one way rather than another. Even questions without bias may leave respondents with expectations.
The order or "natural" grouping of questions is often relevant. Prior previous questions may bias later questions.
The wording should be kept simple: no technical or specialized words.
The meaning should be clear. Ambiguous words, equivocal sentence structures and negatives may cause misunderstanding, possibly invalidating questionnaire results. Double negatives should be reworded as positives.
If a survey question actually contains more than one issue, the researcher will not know which one the respondent is answering. Care should be taken to ask one question at a time.
The list of possible responses should be collectively exhaustive. Respondents should not find themselves with no category that fits their situation. One solution is to use a final category for "other ________".
The possible responses should also be mutually exclusive. Categories should not overlap. Respondents should not find themselves in more than one category, for example in both the "married" category and the "single" category – there may be need for separate questions on marital status and living situation.
Writing style should be conversational, yet concise and accurate and appropriate to the target audience.
Many people will not answer personal or intimate questions. For this reason, questions about age, income, marital status, etc. are generally placed at the end of the survey. This way, even if the respondent refuses to answer these "personal" questions, he/she will have already answered the research questions.
"Loaded" questions evoke emotional responses and may skew results.
Presentation of the questions on the page (or computer screen) and use of white space, colors, pictures, charts, or other graphics may affect respondent's interest or distract from the questions.
Numbering of questions may be helpful.
Questionnaires can be administered by research staff, by volunteers or self-administered by the respondents. Clear, detailed instructions are needed in either case, matching the needs of each audience.
There are a number of channels, or modes, that can be used to administer a questionnaire. Each has strengths and weaknesses, and therefore a researcher will generally need to tailor their questionnaire to the modes they will be using. For example, a questionnaire designed to be filled out on paper may not operate in the same way when administered over the phone. These mode effects may be substantial enough that they threaten the validity of the research.
Using multiple modes can also improve access to the population of interest when some members have different access, or have particular preferences.
- Contingency question – A question that is answered only if the respondent gives a particular response to a previous question. This avoids asking questions of people that do not apply to them (for example, asking men if they have ever been pregnant).
- Matrix questions – Identical response categories are assigned to multiple questions. The questions are placed one under the other, forming a matrix with response categories along the top and a list of questions down the side. This is an efficient use of page space and respondents’ time.
- Closed-ended questions – Respondents’ answers are limited to a fixed set of responses. Most scales are closed-ended. Other types of closed-ended questions include:
- Yes/no questions – The respondent answers with a "yes" or a "no".
- Multiple choice – The respondent has several option from which to choose.
- Scaled questions – Responses are graded on a continuum (example : rate the appearance of the product on a scale from 1 to 10, with 10 being the most preferred appearance). Examples of types of scales include the Likert scale, semantic differential scale, and rank-order scale (See scale for a complete list of scaling techniques.).
- Open-ended questions – No options or predefined categories are suggested. The respondent supplies their own answer without being constrained by a fixed set of possible responses. Examples of types of open-ended questions include:
- Completely unstructured – For example, "What is your opinion on questionnaires?"
- Word association – Words are presented and the respondent mentions the first word that comes to mind.
- Sentence completion – Respondents complete an incomplete sentence. For example, "The most important consideration in my decision to buy a new house is . . ."
- Story completion – Respondents complete an incomplete story.
- Picture completion – Respondents fill in an empty conversation balloon.
- Thematic apperception test – Respondents explain a picture or make up a story about what they think is happening in the picture
The way that a question is phrased can have a large impact on how a research participant will answer the question. Thus, survey researchers must be conscious of their wording when writing survey questions. It is important for researchers to keep in mind that different individuals, cultures, and subcultures can interpret certain words and phrases differently from one another. There are two different types of questions that survey researchers use when writing a questionnaire: free response questions and closed questions. Free response questions are open-ended, whereas closed questions are usually multiple choice. Free response questions are beneficial because they allow the responder greater flexibility, but they are also very difficult to record and score, requiring extensive coding. Contrastingly, closed questions can be scored and coded much easier, but they diminish expressivity and spontaneity of the responder. In general, the vocabulary of the questions should be very simple and direct, and most should be less than twenty words. Each question should be edited for "readability" and should avoid leading or loaded questions. Finally, if multiple items are being used to measure one construct, the wording of some of the items should be worded in the opposite direction to evade response bias.
A respondent's answer to an open-ended question can be coded into a response scale afterwards, or analysed using more qualitative methods.
Questions should flow logically from one to the next.
The researcher must ensure that the answer to a question is not influenced by previous questions.
Questions should flow from the more general to the more specific.
Questions should flow from the least sensitive to the most sensitive.
Questions should flow from factual and behavioral questions to attitudinal and opinion questions.
Questions should flow from unaided to aided questions.
According to the three stage theory (also called the sandwich theory), initial questions should be screening and rapport questions. Then in the second stage you ask all the product specific questions. In the last stage you ask demographic questions.
Computer-assisted telephone interviewing
Computer-assisted personal interviewing
Automated computer telephone interviewing
Official statistics
Bureau of Labor Statistics
Questionnaires
Questionnaire construction
Paid survey
Data mining
NIPO software
DIY research
SPSS
Marketing
Marketing research
Scale (social sciences)
Statistical survey
Quantitative marketing research
