220 likes | 444 Views
Surveys. Sherry Woosley & Cindy Miller. Surveys. Overall. Survey Planning. What are you trying to accomplish?. Surveys should be used to :. Explore attitudes, opinions, experiences , expectations, and needs Gather information from and about large populations
E N D
Surveys Sherry Woosley & Cindy Miller
Surveys Overall
Survey Planning What are you trying to accomplish?
Surveys should be used to: • Explore attitudes, opinions, experiences, expectations, and needs • Gather information from and about large populations • Make comparisons among subgroups of the population • Compare results from year to year • Gather statistically representative data
Surveys should not be used: • For audiences that are uncomfortable with numbers/statistics • For small numbers of participants • Without a clear understanding of the issues • When investigating issues of a sensitive or intrusive nature
Advantages • Can gather information from large numbers • Results may be generalizableto a larger population • Allow for statistical analysis that examines relationships among variables
Disadvantages • Important issues can be overlooked on surveys when the questions and responses are predetermined. • The quality of survey data is strongly dependent on the survey design. • Response rates and response bias are difficult to control.
Surveys Frequently Asked Questions – Other Big Picture Issues
FAQ – What about an on-line survey? • To decide think about: • Who is in your population? • What is in your survey? • Do students care about this topic? • How long is the survey? • What opportunities do you have to administer the survey? • What resources do you have? On-line surveys do NOT necessarily have lower response rates.
FAQ – How many respondents do I need? • The important issues are: • What percentage of people didn’t respond? • How well do your respondents match your non-respondents? • Do you have enough respondents to do the analysis you want? There is no magic number! In most cases, a sample size calculator does NOT adequately answer this question.
FAQ – What types of analysis are appropriate? • Descriptives: Looking at how students responded • Differences: Comparing groups of students • Relationships: Looking at which variables may be related • Change: Looking longitudinally The type of analysis depends on (1) the questions being asked and (2) the statistical skills of the researcher.
FAQ – What is included in a survey report? • Need or problem and research questions • Survey characteristics • Survey methods (administration, sample, analysis) • Results • Conclusions • Implications Audience needs will determine the level of detail included in a report.
Surveys Questions / Items
Good Survey Questions • Wording should be simple, clear, direct, non-ambiguous, concrete, and uniformly understood. • There is no one “correct” response scale for all survey items.
Question Types • Open ended • Single response • Essay/paragraph • Closed • Categorical (categories) • Scales (Likert) • Rankings • Choose multiples or choose only one
Good Surveys… • Use neutral language • Generate a variety of responses • Use simple sentences (avoid compound sentences and multiple phrases) • Consist of only one question (beware of double-barreled questions)
Bad Surveys… • Ask things you already know • Ask things that your respondents cannot answer • Be too intrusive or personal • Include • universals (always, all, none) • limiters (only, just) • double negatives • abbreviations • unconventional phrases
Bad Examples – Can you identify what went wrong? • How many times did you call your parents last year? • How much time does your teacher spend preparing for class? • How often do you eat donuts and/or drink coffee? • How often have you been harassed on this campus? • How often have you participated in ACCESS? • How often are you merely late for a class? • How often do you miss class and feel bad about it afterwards? • How often do you never miss a class?
Strategies to Ensure Good Questions • Have the questions reviewed by experts • Have the questions reviewed by potential respondents • Have the questions reviewed by colleagues • Adopt or adapt questions that have been used successfully on other surveys • Pilot test surveys
Discussion Your turn… • What did you bring? • What questions do you still have? • How can you share your expertise with others?
Questions? • Thanks for attending today’s session. • More questions? Feel free to contact us. Cindy Miller cmiller@bsu.edu 285-8013 Sherry Woosley sawoosley@bsu.edu 285-5976