Survey Development and Design

1. Survey Development and Design Session Three EDUC 611

2. Surveys Writing a useful survey is much tougher than you think! • Surveys are a mechanism for obtaining quantifiable data that address your objectives.

3. List Your Objectives • The survey should be guided by your objectives. What is your purpose in creating the survey – what do you want to find out? • Each survey item should deal with an objective. • A common mistake: Gathering data you didn’t need and can’t use.

4. Achieve Clarity • Principles of Good Survey Writing: • Simple words • Short, direct statements • Easy-to-use scales • Explicit meaning

5. Ensure Proper Flow • Introduction: • Start with an explanation of purpose and clear directions. • Early questions are usually the easiest to answer. • “Like” items should be grouped together. • Multiple choice/Likert scale are easiest to score, but open-ended questions often give the most unexpected, enlightening information. • Avoid giving the “other” option if possible (unless you aks for an explanation.) • Demographic information usually goes at the end.

6. Generate Credibility • Document should look clean, uncluttered, and professional. • Use of white space is very important. • Varying fonts across instructions and questions can help. • Using sections, subheadings, and other organizing practices is a plus.

7. Survey the Right People • You may decide to use a screener. • (This allows you to give a full survey only to those who qualify as members of your population of interest.) • e.g.: Are you a college student? __yes __no

8. Tips • Depending on the size of your population, you may want to pretest your survey. • (A pretest is a trial run.) • Make sure pretest subjects provide comments on wording, meaning, grammar, etc. • Assume you will edit a first draft.

9. Open-ended vs. Closed-ended Questions • Open-ended questions are questions to which there is not one definite answer. • Open-ended questions may be a good way to break the ice with a survey, giving respondents an opportunity to answer in their own words. • e.g.: "Are there any other comments about the course you would like to add?" • The drawback to open-ended questions is that the responses are more difficult to catalogue and interpret. • http://coe.sdsu.edu/eet/Articles/surveyquest/index.htm • Fink, A. (1995). How to ask survey questions. Thousand Oaks, CA: SAGE Publications.

10. Open-ended vs. Closed-ended Questions cont. • Closed-ended questions have a finite set of answers from which the respondent chooses. • The benefit of closed-ended questions is that they are easy to standardize, and data gathered from closed-ended questions lend themselves to statistical analysis. • The down side to closed-ended questions is that they are more difficult to write than open-ended questions. This is because the evaluator must design choices to include all the possible answers a respondent could give for each question. • http://coe.sdsu.edu/eet/Articles/surveyquest/index.htm • Fink, A. (1995). How to ask survey questions. Thousand Oaks, CA: SAGE Publications.

11. Likert Scales • The participants must indicate how closely their feelings match the question or statement on a rating scale. • The number at one end of the scale represents least agreement, or "Strongly Disagree," and the number at the other end of the scale represents most agreement, or "Strongly Agree." How important do you think standardized test scores are to a fifth-grader's education (circle one number): Very ImportantNot Very Important 1 2 3 4 5 • http://coe.sdsu.edu/eet/Articles/surveyquest/index.htm • Fink, A. (1995). How to ask survey questions. Thousand Oaks, CA: SAGE Publications.

12. Happy Dale High School Mathematics Department • Please check in the appropriate space below: • ____ Male ____ Female • ____ Freshman ____ Sophomore ____ Junior ____ Senior • This scale has been prepared so that you can indicate how you feel about the statements below. Please circle one of the letters on the left indicating how you feel about each statement (SA, strongly agree; A. agree; D.disagree; SD. strongly disagree). • SA A D SD 1. I understand explanations given by my teacher. • SA A D SD 2. I can see the board and overhead screen clearly. • SA A D SD 3. When something is unclear, I am not comfortable asking a question in class. • SA A D SD 4. Being in a class taught by two teachers helps me to be more attentive. • SA A D SD 5. I am not doing very well in this class. • SA A D SD 6. My teacher is available for extra help. • SA A D SD 7. I am not comfortable participating in this class by responding to teacher questions. • SA A D SD 8. I am confused by explanations given by my teacher(s). • SA A D SD 9. It’s hard for me to see the board and overhead screen from where I’m seated. • SA A D SD 10. I am comfortable asking a question in class if I don’t understand something. • SA A D SD 11. I find it harder to pay attention in class with two teachers. • SA A D SD 12. I am doing pretty well in this class • SA A D SD 13. My teacher is not available when I need extra help outside the classroom. • SA A D SD 14. I am comfortable participating in class by responding to teacher questions.

13. Multiple Choice • When you want the participants to pick the best answer or answers from among all the possible options. • Include specific directions about how many answers to select directly after the question. e.g.: • Why don't you use the school's cafeteria services? (circle one): • a. It's too expensive. • b. Serving times conflict with my class schedule. • c. The location is inconvenient. • d. The food quality is poor. • e. Other (please explain):_______________ • http://coe.sdsu.edu/eet/Articles/surveyquest/index.htm • Fink, A. (1995). How to ask survey questions. Thousand Oaks, CA: SAGE Publications.

14. Ordinal • When you need all possible answers to be rank-ordered. “Please write a number between 1 and 5 next to each item below. Put a 1 next to the item that is MOST important to you in selecting an on-line university course. Put a 5 next to the item that is LEAST important. Please use each number only ONCE.___” • a. Availability of instructor for assistance.___ • b. Tuition cost for the course.___ • c. Ability to work in groups with other students.___ • d. Quality and quantity of instructor feedback.___ • e. Number of students enrolled. • http://coe.sdsu.edu/eet/Articles/surveyquest/index.htm • Fink, A. (1995). How to ask survey questions. Thousand Oaks, CA: SAGE Publications.

15. Checklist • Do you have objectives? • Have you asked questions that will address them? • Are your prospective survey participants appropriate? • Are the questions reasonable? (i.e., how many bowls of cereal did you eat last year? What were the room numbers of your classes last semester?) • Are the questions clear and easy to understand? • Is the survey brief and concise? • http://www.jou.ufl.edu/people/faculty/mweigold/adv3501/Designing%20a%20survey(l9)_files/frame.htm

16. Sources Information for this presentation obtained from: • http://coe.sdsu.edu/eet/Articles/surveyquest/index.htm • http://www.jou.ufl.edu/people/faculty/mweigold/adv3501/Designing%20a%20survey(l9)_files/frame.htm