Survey Research: Sampling Designs, Question Construction, and Interviewing Techniques

1. URBP 204 A Class 9 • CLASS 9 • Correlation Coefficient • Tutorial 3 • Give back Rev Ex Set 1 and Neighborhood Profile Memo • Survey Research • CLASS 10 • Probability- based Sampling • Survey Research • Experiments (Time Permitting) • Qualitative Field Research (Time Permitting) • Introduce Mid-Term • Introduce Exercise Set 3 • Factorial ANOVA Example • Lab-Time for Term Project Note: the class notes summarize Salkind (2004) Chapters 12, 13 and 15

2. Types of probability sampling designs Simple random sampling All elements are numbered and then a predetermined number are randomly selected. (may get clusters, should know the entire population) Systematic sampling Every kth element chosen For e.g.: every 10th house; danger of biases (every 10th house may be a corner house - traffic noise) Stratified Sampling Reduce sampling error by less variability e.g. single-family houses and apartments Within the strata we may employ simple random or systematic sampling Multistage clustering sampling Sampling of clusters City-wide study - list census tracts - choose sample of census tracts - list all block groups - choose sample of block groups - list all blocks - choose sample of blocks - list all h.h.s - choose sample of h.h.s May also stratify the sample: Stratification in multistage cluster sampling Note: the class notes summarize Babbie (2004) Chapter 7, and part of Chapter 9.

3. Survey Research • Guidelines for asking questions • Open ended vs close ended question • Open ended – more rich; coding difficult; more chances of error while recording • Close ended – less rich; responses may be straight jacketed; less chances of recording • error; careful about exhaustiveness • Make items clear - income last year – based on W-2? • Avoid double-barreled questions – Should San Jose cut back on road • construction and increase allocation for affordable housing? • Respondent must be competent and willing to answer • Questions should be relevant • Keep it short Note: the class notes summarize Babbie (2004) Chapter 7, and part of Chapter 9.

4. Guidelines for asking questions contd…. • Avoid negative terms – do you think we should not do this? • Avoid biased items and terms – do you agree with the recent health reports’ finding that…… Note: the class notes summarize Babbie (2004) Chapter 7, and part of Chapter 9.

5. Guidelines for survey interviewing • Modest yet neat appearance • Avoid voice inflections • Be neutral • Be polite • Be familiar with the questions • Don’t add your words to the question • Record responses exactly • Gently probe for responses/ clarifications Note: the class notes summarize Babbie (2004) Chapter 7, and part of Chapter 9.

6. Questionnaire Construction • Not cluttered • Professional look; clear instructions on how to choose responses (especially if self-administered), • Use of contingency questions • Use of matrix format • Order of questions – bias due to the order (negative aspects of sprawl; then ask is sprawl bad); in self- administered begin with interesting questions; in interviews – uncomplicated questions first • Include clear instructions and introductory statements (now think about the last one week….) • Pretest the questionnaire

7. Self- administered Questionnaire Mail; home delivery or combination; questionnaire at a public gathering; etc. If mail delivery Monitor the return Follow up mailings Telephone Surveys Unlisted number – random digit dialing Advantages – cheap and quick; no dress code; probe more sensitive areas; more quality control possible as central location; safety. Disadvantages – compete with bogus surveys; easy to hang up; answering machines; cell phones

8. Comparison of self-administered and interview survey methods • Advantages of self- administered over face-to-face interviews • Cheaper • More geographically extensive • Require smaller staff • Easy to probe sensitive topics • Advantages of face-to-face interviews over self- administered questionnaires • Larger response rate • More effective for complicated issues • May also note other information - condition of the neighborhood, etc.

9. Strengths and weaknesses of survey research • Strengths • Describe characteristics of a large population • Make large samples feasible • More flexible – can cover several topics • Strong on reliability • Weakness • Have to ascribe the same intent to responses to questions related to complex concepts • Least common denominator – superficial in coverage of complex topics • Life situation/ context not known • Can not change questions mid-way • Respondent may form opinion at the moment • Weak on validity

10. MODES OF OBSERVATION • Experiments • Surveys • Qualitative Field Research • Others ( for example, content analysis)

11. EXPERIMENTS Taking action Observing consequences of the action Suited to : a) Research “well-defined concepts and propositions” (Babbie 221) b) Explanatory studies c) Small groups Can be: Laboratory experiments (health sciences, physical sciences, etc.) Natural experiments (naturally occurring events)

12. Components of a classical experiment • Dependent and independent variables • Pretesting and posttesting • Experimental and control groups • Effect of knowledge of good examples of high density development • on the people’s perception about density • Selecting subjects • Randomization • Matching (Income, urban vs rural, race, etc.) - like quota sampling • Combination of both, akin to stratified sampling

13. Variations in Experimental Design • Preexperimental research designs • One shot case study – no pretest – just show examples of good high density developments and then measure the perception • One- group pretest- posttest design – no control group – not sure whether examples caused the effect • Static-group comparison – no pretest; control group present- randomization of subjects important (examples of good high density development showed to one group and not to the other)

14. Validity Issues in Experimental Research Internal validity issues- does the conclusion reflect the process? History – major event during the experiment Maturation – people old, tired, sleepy. Slept while shown examples of high density development! Testing – testing and retesting affects subjects’responses Instrumentation- pretest and posttest measures different Statistical regression- extreme positions undesirable as these people are likely to improve anyways Selection biases – groups should be comparable Experimental mortality – bias due to subjects leaving Causal time order – dependent variable causes changes in the stimulus – e.g. rat causes change in virus; Diffusion of treatments- experimental and control groups in communication Compensation – to the control group. Thus they are no longer ideal control group. Compensatory rivalry – extra efforts by the control group Demoralization – of the control group

15. External Validity Issues in Experimental Research Is the experiment generalizable? What if instead of slides of good high density development we showed a film? Natural experiments Natural events as stimulus- e.g. hurricanes, bombings, etc. Some of the Limitations: Comparable control groups difficult to find Selection of subjects not random

16. Strengths and Weaknesses of the Experimental • Method • Strengths • Replication is easy – more reliability • Interaction of dependent and independent variable easy to isolate • Weaknesses • Artificiality- changes may not occur in more natural settings

17. Qualitative Field Research • Topics appropriate for Field Research • Hard to quantify • Nuances • Important to observe in natural settings • Social processes over time • Study of relationships, roles, groups, subcultures, lifestyles, etc. • Special Role of Observers • Complete observer – does not participate at all. Less appreciation of • what is being studied. • Complete participant – full participation – genuine or pretend; for • subjects not a researcher. Issue of deception; affecting the study; going • native! • Somewhere in between- participate but let the subjects know that you • are a researcher

18. Relations to subject Pretend to join or really join - adopt the subjects’ point of view or view them critically- or combination The setting can have an impact – e.g. homeless on the streets, welfare office, or shelter Swayed by subjects’ personal lives (problems, etc.) - threat of loosing scientific objectivity Notion of superiority over the subjects

19. Strengths and Weaknesses of Qualitative Field Research Strengths Not superficial Flexibility- research design; begin the observation Can be inexpensive Greater validity than survey and experimental design Weaknesses Can’t statistically describe a large population Not good at predicting future Less reliability than the survey and experimental design