Developing Basic Skills for Conducting Simple Surveys

1. Developing Basic Skills for Conducting Simple Surveys Adrian F. Aveni Professor Department of Sociology and Social Work Jacksonville State University Jacksonville, Alabama USA

2. Information About Our Course Internet Address: https://www.coursesites.com/ Name: CourseSites Course: Basic Skills for Simple Surveys School: Jacksonville State University Instructor: Adrian Aveni Other basic information is required

3. Outline of Topics-1 • Overview of the Workshop • Identify the research objectives • Determine the level of accuracy needed • Determine the type of data needed to answer the research questions • Identify all of the variables to be included in the study • Sampling designs: types of samples; sampling sites • Data collection instruments: questionnaires; interview schedules; interview guides • Types of questions in surveys • How to obtain questions for the study • Determine the feasibility of the study: time/money/expected outcomes • Principles of constructing interview schedules and questionnaires

4. Outline of Topics-2 • Ethical Considerations • Interviewing • Data collection • Developing a coding scheme • Coding data into a data file • Cleaning data • Statistical software alternatives • Describing the findings • Comparing the sample and population characteristics • Analyzing the data • Writing up the research report • Characteristics of a good report

5. Tentative Time Table of the Workshop Monday: AM: Identify Research Objectives -> Probability Samples Monday: PM: Non-Probability Samples -> Questionnaires Tuesday: AM: Interview Schedules -> Interviewing Tuesday: PM: (Plan a Research Project: Sampling Design; Data Collection Instrument; Coding Scheme) Wednesday: no class: (Data Collection: Conduct a Survey; Code Data) Thursday: AM: (Compile Data); Cleaning Data -> Comparing Sample and Population Characteristics Thursday: PM: Performing More Detailed Analyses -> Characteristics of a Good Report Friday: AM: (Analyze Data; Compare Sample and Population Characteristics; PM: Begin Writing A Report)

6. Identify the Research Objectives • What do you want to learn? • Common Research Categories: • Academic • Government/Private Sector • Informal

7. Issues of Accuracy • Question: What Level of Accuracy Is Required • Highly vs. Moderately vs. Coarsely Accurate Studies • Not All Studies Can Be Highly Accurate • Not All Studies Need to Be Highly Accurate

8. Determine the Type of Data Needed • Question: Is a Survey the Best Instrument? • Alternatives to Surveys • Focus Groups • Observational Research • Talking with “Knowledgeables”

9. Identify the Variables to be Studied • Common Ways of Organizing Variables • Primary Research Variables (Dependent Variables) • Possible Causal Variables (Independent Variables) • Background Variables • (e.g. age, education, occupation, gender…) • Rationale

10. Political Variables eligible to vote plan to vote political party liberalism-conservatism volunteered for a campaign/candidate able to identify Republican candidates Religious Variables religious preference religiosity Student Characteristics major year in school grade point average student organization memberships Background Variables age marital status region of the country city size gender race Example: Political Survey

11. CHARACTERISTICS OF INTERNATIONALIZATION Programs listened to campus (92J) radio programs “Latin music” Thistle & Shamrock visited the International House lived in a residence hall with international students on your floor involvement in activities/clubs that included international students Curricular studied a foreign language at JSU number of semesters of study number of courses entirely international in focus/content number of courses with some international focus/content CHARACTERISTICS OF INTERNATIONALIZATION Students taken any courses in another country traveled at any time in another country university-related living or employment foreign exchange student living with your family do you know any international students on campus do you have any international students in your classes do you live with any international students Faculty have you had any teachers from other countries have you had any teachers who traveled/worked/lived/ in other countries Example: Internationalization Survey (p. 1)

12. CONSEQUENCES OF INTERNATIONALIZATION knowledge about other countries country capitals names of political leaders interest in future work/travel abroad desire to take a vacation abroad desire to study abroad desire to work abroad openness to cross-cultural experiences seen a foreign film last year listen to foreign music ethnocentrism BACKGROUND VARIABLES age education race/ethnicity gender major year in school GPA number of years at JSU mother’s occupation father’s occupation parents’ income parents’ education PRIOR CHARACTERISTICS parents travel abroad studied a foreign language in high school studied a foreign language outside of high school and JSU Example: Internationalization Survey (p. 2)

13. Sampling Design • Question: At What Location Will People Be Contacted? • Question: How Many People Will Be Sampled? • Question: What Type of Sample Will Be Used?

14. Some Terms Population – refers to ALL of whatever you are studying can be people; cars; magazines; songs; anything Elements – refers to the units being studied within the population e.g. if the population is all students at UFV, the elements are students Census – is a study of ALL of the elements of the population Sample – is a study of SOME of the elements of the population

15. Two Basic Types of Samples • Probability Samples Common Types: Random; Stratified; Cluster • Use random procedures in selecting people • Advantages • Disadvantages • Non-probability Samples Common Types: Systematic; Quota; Accidental • Do not use random sampling procedures • Advantages • Disadvantages

16. Random Sampling Requirements 1. Clear definition of the population 2. Complete listing of all elements in the population 3. Assumption: all elements are ‘statistically independent’ Steps 1. Obtain a listing of all the elements in the population 2. Number all the elements in the population, from 1 to N 3. Decide upon the sample size 4. Draw random numbers 5. Those elements in the population with numbers matching the numbers drawn are selected as part of the sample 6. Continue drawing numbers until the desired sample size is obtained 7. If possible, compare the sample and population characteristics

17. Stratified Sampling Requirements: 1. Clear definition of the population 2. Complete listing of all elements 3. Assumption: all elements are ‘statistically independent’ 4. Must be able to identify one or more variables/criteria to stratify on (these variables must be theoretically relevant) 5. Must be able to identify the variables ahead of time

18. Stratified Sampling Steps in Drawing a Stratified Sample: 1. Obtain a listing of all the elements in the population 2. Identify relevant variables to use in stratifying 3. Divide the population up into the relevant strata 2. Number all the elements in each strata, from 1 to N 3. Decide upon the sample size to be taken for each strata 4. Draw numbers for each strata 5. Those elements in the population with numbers matching the numbers drawn are selected as part of the sample for each strata 6. Continue drawing numbers until the desired sample size is obtained for each strata 7. If possible compare the sample and population characteristics

19. Cluster Sampling Overview: Cluster sampling is based on the view that often it is easier to sample people who are gathered together in groups (or clusters) rather than one at a time Requirements: • similar requirements to random sampling • instead of having to identify all of the elements of the population, you only have to identify all of the clusters in the population 1. Clear definition of the population 2. Complete listing of all the clusters in the population 3. Assumption: all of the clusters are ‘statistically independent’ of one another

20. Variations in Cluster Sampling • Multi-stage Cluster Sampling: • a term used to describe a sampling design where clusters are located within clusters • Block Sampling • a term used to describe sampling where the clusters are city blocks • Using Cluster and Stratified Sampling Together

21. Systematic Sampling Requirements: 1. Clear definition of the population to be studied 2. Complete listing of all elements (There is NO assumption of statistical independence)

22. Systematic Sampling Steps in Drawing a Systematic Sample 1. Obtain a listing of all the elements in the population 2. Number all the elements in the population, from 1 to N 3. Decide upon the sample size 4. Divide the population size by the sample size this number is the “Sampling Interval” Example: population is 10,000 and sample is 1,000 sampling interval is 10: every 10th person will be selected 5. Take a “random start” (i.e. randomly choose a number from 1 to the value of the sampling interval) 6. Select the case with the number corresponding to the random start 7. Proceed to select cases according to “sampling interval” 8. When you have reached the end of the list you will have the desired sample size

23. Quota Sampling Overview • the researcher must identify key population characteristics e.g. age; gender; income; political affiliation • objective is to draw a sample that has the same proportion of persons as the population on the key characteristics • interviewers are given “quotas” of the types of people they should interview in this respect it is like stratified sampling

24. Steps in Drawing a Quota Sample 1. Identify key variables in the population to be studied 2. Identify the distribution of persons in the population on these key variables 3. Determine the sample size needed 4. Send interviewers out into the community with quotas on the key variables 5. Where the completed interviews do not completely match the population characteristics, the researcher “weights” the interviews to match those of the population

25. Accidental Sampling There are no requirements! Steps: 1. The researcher begins sampling from the population: in whatever location is most convenient and whichever people happen to be there 2. The researcher continues drawing a sample from that location, or from other convenient locations, until the desired sample size is reached

26. Estimating Required Sample Sizein Probability Samples sample size for:pop sizemargin of errorconfidence level 377 20,000 +/- 5% 95% 370 10,000 278 1,000 218 500 152 250 80 100 45 50 1333 10,000 +/-2.5% 95%

27. Example #1: Issues to Consider in Sampling Problem: How to Sample College Students Some Alternatives: • mailing questionnaires to campus addresses • e-mailing questionnaires to their campus e-mail address • distributing questionnaires in class • distributing questionnaires in booths around campus • calling their campus phone numbers and interviewing • visiting them at their residences and interviewing • contacting outside of classrooms and interviewing

28. How Many College Students to Sample? Campus population is about 10,000 not possible or practical to include everyone hoped for a sample size of 456 38 students x 12 interviews/student = 456 obtained a sample of 420 .

29. Example #2: Issues to Consider in Sampling Problem: How to Sample Public Housing Residents Situation: Residents were in apartments in five different locations Alternatives: mail questionnaires to residents ask Housing Authority to distribute questionnaires interview residents by phone interview residents at their apartments

30. How Many Public Housing Residents to Sample? Issues: did not know exactly how many residents identified 211 different apartments Solution: decided to take a census contacted 1 person from each apartment used areal maps to identify apartment buildings counted mailboxes and located apartments assigned interviewers to apartments

31. Questionnaires • A set of questions that are given to a respondent to answer him/herself • Are “self-administered” • The questions are in a fixed order (can’t change the order of the questions) • Possible ways of distributing questionnaires: hand out in-person to individuals hand out in-person to groups (most exams) mailed out to individuals sent via e-mail to individuals

32. Questionnaires strengths : 1. generally less expensive and time consuming than interviews 2. may help give respondents a feeling of anonymity (they are not known) 3. good for factual information; and attitudes drawbacks: 1. often they have higher levels of error than interviews 2. the response rates are usually lower than for interviews 3. respondents must be literate

33. Interview Schedules • a set of questions that an interviewer asks a respondent • either face-to-face or over the phone • the questions are in a fixed order (can’t change the order of the questions) • the interviewer records the answers to the questions on the interview schedule or, directly onto a computer

34. Interview Schedules Strengths: 1. generally provide a more accurate description of the population 2. complex questions can be asked in interviews 3. questions that are very general and require extensive answers can be asked 4. very personal questions can be asked in interviews drawbacks: 1. more expensive and time-consuming than questionnaires 2. require selection, training, and monitoring of interviewers

35. Interview Guides • a set of points or topics that an interviewer wishes to cover during an interview • the points/topics can be very broad or general • they can be discussed in any order • more commonly used in anthropological research and in participant observation research in sociology

36. Items in Surveys • an item is a question asked in a survey • item structure refers to: • how specific (restrictive) is the question being asked in an item; and • how specific (restrictive) are the possible answers to the question • item structure falls along a continuum from highly structured items to loosely structured • Open-ended items allows the respondent to answer any way he/she would like (loosely structured) • Closed-ended items restrict the possible answers the respondent may give (highly structured)

37. General Rules for Using Items • Whenever possible, use closed-ended items in surveys • Closed-ended items are best used for gathering: • factual information (e.g. age, education level, year in school) • information about past behavior (e.g. frequency of church attendance) • attitudes on various topics • Closed-ended items are not good when: • you need information on complex issues • you study how people arrived at their decisions • when you don’t know the relevant categories

38. How to Obtain Items for Your Survey Two Options: Obtain Items from Other Studies Constructing Items Yourself General Rules: For serious research: whenever possible use items from other studies; For informal research: construct items yourself

39. Steps Used When Constructing Your Own Items 1. conduct a pilot study begin with open-ended items on the subject use an “interview guide” take a small sample from the population administer the interview guide perform a “content analysis” on the items develop closed-ended items 2. perform a pre-test on your Q/IS take another small sample administer the questionnaire/interview schedule evaluate the items

40. Determine the Feasibility of Your Study Given: your research objectives the necessary accuracy the type of data you need the type of sample you want the data collection instrument you want Is your study feasible? IF “yes” – then proceed IF “no” – then reconsider your plans/options

41. Example: Lack of Proper Planning Purchasing Managers Survey • joint research with colleague in the business colleague • the colleague was a good businessman but not good at surveys • the colleague asked me to help with questionnaire design • I spent many hours developing and refining a questionnaire • The research design required permission from the head of a professional organization of Purchasing Managers who would give the names and addresses of the managers • my colleague knew who the person was and how to contact him • colleague neglected to contact the head before hand • when he did contact the head the person refused • our study was terminated after many hours of work on the questionnaire

42. Some Differences Between Questionnaires and Interview Schedules • Questionnaires usually have longer introductions • Mailed questionnaires usually have an accompanying letter • The language is more formal/proper in questionnaires • Questionnaires may require greater care in selecting items to include • The format and presentation of the questionnaire must look “professional”

43. See Penn State Questionnaire .

44. See Wall Street Journal Questionnaire .

45. Principles of Constructing Interview Schedules and Questionnaires (ISs & Qs) • Give ID Numbers to all ISs and Qs • All Questions Should be Numbered • Record the Name of the Interviewer • Record the Date, Time, and Location • Have an Introduction to the IS or Q • Use Simple Sentences and Basic Vocabulary • Keep Similar Items Together • Begin with Easy-to-Answer Items Directly Related • Gradually Increase Difficulty • Have Most Difficult in the Middle • End with Easy-to-Answer Items Not Directly Related • Thank People for Participating • Ask for Comments or Suggestions

46. DAY TWO • .

47. Information About Our Course Internet Address: https://www.coursesites.com/ Name: CourseSites Course: Basic Skills for Simple Surveys School: Jacksonville State University Instructor: Adrian Aveni Other basic information is required

48. REVIEW • Clear Objectives • Determine Level of Accuracy • Determine Type of Data Needed • Identify All Variables to be Studied • Choose the Type of Sample • Determine Sample Size • Determine Sample Location • Determine Type of Instrument (Q/IS) • Determine Feasibility • Construct the Instrument

49. Discussion and Clarification #1 • Response Rate = the number of persons who answer a survey the number of persons contacted • The lower the Response Rate the less representative the survey is of the population

50. Discussion and Clarification #2 Determining Sample Size **For probability samples, use statistics showing Margin of Error for a given sample size sample size for:pop sizemargin of errorconfidence level 377 20,000 +/- 5% 95% **For all samples, identify the largest table you need for analysis look at row and column totals calculate expected cell frequencies