QUESTIONNAIRE DESIGN: GENERAL PRINCIPLES

1. QUESTIONNAIRE DESIGN: GENERAL PRINCIPLES Lu Ann Aday, Ph.D. The University of Texas School of Public Health

2. MEASUREMENT: Definition of Variables

3. MEASUREMENT: Levels of Measurement

4. MEASUREMENT: Example Concept:Obesity

5. MEASUREMENT: Example Concept:Family Income

6. PRINCIPLES FOR WRITING QUESTIONS (Dillman, 2000) • 1. Choose simple over specialized words. • 2. Choose as few words as possible to pose the question. • 3. Use complete sentences to ask questions. • 4. Avoid vague quantifiers when more precise estimates can be obtained.

7. PRINCIPLES FOR WRITING QUESTIONS (Dillman, 2000) • 5. Avoid specificity that exceeds the respondent’s potential for having an accurate, ready-made answer. • 6. Use equal numbers of positive and negative categories for scalar questions. • 7. Distinguish undecided from neutral by placement at the end of the scale.

8. PRINCIPLES FOR WRITING QUESTIONS (Dillman, 2000) • 8. Avoid bias from unequal comparisons. • 9. State both sides of attitude scales in the question stems. • 10. Eliminate check-all-that-apply question formats to reduce primacy effects. • 11. Develop response categories that are mutually exclusive.

9. PRINCIPLES FOR WRITING QUESTIONS (Dillman, 2000) • 12. Use cognitive design techniques to improve recall. • 13. Provide appropriate time referents. • 14. Be sure each question is technically accurate. • 15. Choose question wordings that allow essential comparisons to be made with previously collected data.

10. PRINCIPLES FOR WRITING QUESTIONS (Dillman, 2000) • 16. Avoid asking respondents to say yes when they mean no. • 17. Avoid double-barreled questions. • 18. Soften the impact of potentially objectionable questions. • 19. Avoid asking respondents to make unnecessary calculations.

11. SCALES: Definition • Scales are measurement instruments that are collections of items [questions/statements] intended to reveal theoretical variables [abstract concepts] not readily observable by direct means, e.g., patient satisfaction, locus of control.

12. SCALES:Underlying Structure • ABSTRACT CONCEPT, e.g., Unsatisfactory Patient Encounters • Specific Items, e.g., series of statements regarding the nature of Patient Encounters • Uniform Response Categories, e.g., scale of 1 to 5, where 1 is very typical and 5 is not at all typical of patient encounters

13. SCALES: Example—Unsatisfactory Patient Encounters(Aday & Cornelius, 2006, NDMS, Q. 3)

14. SCALES:Methodological Issues • ABSTRACT CONCEPT • Internal consistency reliability: Verify that the specific items in the scale relate to the CONCEPT. • Construct validity: Verify that the specific items in the scale define and distinguish the CONCEPT from other concepts.

15. SCALES:Methodological Issues • Specific Items • Take reading level of respondents into account in phrasing items. • Avoid jargon and colloquialisms. • Express only one idea in each item. • Use both positively & negatively worded items, but avoid use of negatives (e.g., “not”) in negatively wording an item.

16. SCALES:Methodological Issues • Uniform Response Categories • Decide on a reasonable number of response categories, e.g., 3-10, taking concept and respondents’ ability to discriminate into account. • Select meaningful labels/terms for categories, e.g., strongly agree, agree, disagree, strongly disagree; excellent, good, fair, poor.

17. MEASUREMENT MATRIX: National Dental Malpractice Survey (Aday & Cornelius, 2006, Table 15.1)

18. SURVEY ERRORS: General Principles for Formulating Questions

19. REFERENCES • DeVellis, Robert F. (2003). Scale Development: Theory and Applications. Second Edition. Thousand Oaks, CA: Sage. • Dillman, Don A. (2000). Mail and Internet Surveys: The Tailored Design Method. Second Edition. New York: John Wiley & Sons, Inc.