90 likes | 148 Views
Survey Research at WCU. Hal Herzog. The truth about the subject pool:. Little incentive to give you good data. Usually young and inexperienced. Often have a haphazard approach to surveys. Compromise the validity and reliability of your results.
E N D
Survey Research at WCU Hal Herzog
The truth about the subject pool: • Little incentive to give you good data. • Usually young and inexperienced. • Often have a haphazard approach to surveys. • Compromise the validity and reliability of your results. • Source of unwanted variation in data (experimental error) • Reduce the effect size of your results
The truth about experimenters: • Often unconsciously give the message that the research is not important. • Show up late. • Don’t show up at all. • Treat subjects unprofessionally. • Undergraduate assistants are not trained. • Sloppy signage on rooms. • Surveys are often poorly designed.
My experience with surveys: • Galvin and Herzog (1998) – animal rights demonstrators • Handed out 750 surveys • Got 231 back (33% return rate) • Scott Plous (1997) – psychologists • Sent surveys to 5,000 APA members • Got 3,982 back (80.4% return rate) • Plous and Herzog (2001) – Animal care committees • Sent 556 surveys to university faculty members • Got 494 back (87% return rate)
Tailored Design MethodDon Dillman • Mail and Internet Surveys: The Tailored Design Method • Based on social exchange theory (Thibaut and Kelly 1950) • It works – increases typical return rates from 25% to 75%. • TDM principles apply to getting better data from WCU Subject Pool • Basic information in Chapters 1 – 3.
Some Principles of Treatment of Subjects:Reward and Reciprocity • Show positive regard • introduce yourself • ask their name • Say “Thank You” – in advance. • Give tangible rewards – in advance. • Certificate of participation? • Make the questions interesting. • Sponsorship by legitimate authority. • Make task appear important. • Ask for advice.
Some Principles of Survey Design: • Booklet format • Cover – have a picture on it. • Introductory letter. • Order of questions. • Pay attention to details – e.g. font size, bold, italics, etc. • Provide a navigational path. • Use back page for open-ended question.
Some Principles for Writing Questions: • Use complete sentences. • Choose simple over specialized words. • Use as few words as possible to pose the question. • Use mutually exclusive response categories. • Separate “undecided” from “neutral” on Likert scale items. • Included an open-ended question at the end.
Other tips. • Consent form should be user-friendly. • Question/answer format? • Pilot studies are critical. • Assess question clarity. • These details make a difference!