Research Methods Psych 402

1. Research MethodsPsych 402 • LECTURE 12 – Survey • Video: Against All Odds #9 (VH 514 E1 part 1 Correlation)

2. Are motor skills impaired when you’re anxious? Are driving skills impaired by marijuana use? Does smoking pot make you less aggressive? Does drinking alcohol make you more aggressive? Are teenagers who smoke more antisocial than teenagers who don’t? Are children of alcoholics more likely to divorce than other groups? Are children of alcoholics more likely to develop mental illness? Does divorce alter a child’s self-esteem? Does teenage sex alter a girl’s self-esteem? Does teenage sex alter a boy’s self-esteem? Does watching violent TV make boys more aggressive? Does watching violent TV make girls more aggressive? Does watching TV impact school performance? Is episodic memory impaired by drinking alcohol? Is semantic memory impaired by drinking alcohol? Are photographic memories true records of events? Is eyewitness testimony reliable? Are women better at perceiving emotions than men? …. Library Query Assignment

3. Assignment 2: Due 4/18 Question: Does divorce alter self-esteem? Answer. Yes. No. maybe (choose one) Constanza (1989) reported….. Kramer and Kramer (1995) reported….. Kramer and Seinfeld (1996).. • Citations

4. Authors (Year). Article title. Journal Title, Volume Number, pages. Roosevelt, F.D., & Truman, H. (1945). Evolutionary strategies in politics. American Journal of Research, 35, 124-138. Authors (Year). Book Title. City published: Publisher. Arnheim, R. (1971). Art and visual perception. Berkeley: University of California Press. APA Citation Format: Journal or Book • Last Name, F. M., Last Name, F.M, and Last Name, F.

7. Observing one (or few) individuals in considerable detail Often unusual examples such as brain injured, serial killers, leaders, extreme abilities Direction observation v secondary analysis (e.g., archival analysis of writing, speeches) Highly informative but difficult to generalize unless ensure representative of case Data open to interpretation (biases, degraded objectivity, calibration shifts, focus on exotic data) Case Study

8. NON-EXPERIMENTAL Research: • Observational • Surveys (interviews or questionnaires) • face-to-face, • mailouts, • telephone, • internet • used most often in ‘social sciences’: • psychology and sociology, • market & policy research • Archival [Correlational research]

9. Non-experimental research often seeks to measure two (or more) variables & look at relationship between them. • Main advantage - Useful when experiments are impractical/unethical • Main disadvantage - Correlation doesn’t prove causation

10. Scatterplot showing relationship between age & self-esteem

11. Size of Correlation (Cohen, 1988) WEAK (.1 - .3) MODERATE (.3-.5) STRONG (>.5)

12. Correlation Checklist Scatterplot • Strength and direction • Outliers? • Linear function?

13. Surveys • Often used in applied (vs basic/pure) research; examine specific “practical” problem or social issue. • can collect both quantitative & qualitative data. • potential variables measured - attitudes, behaviors, public opinion, satisfaction, emotion, stress, etc. – all self-reported, however

14. Example: Stress Survey

15. Survey Research: Goals • Exploratory • is there a relationship between age & spending habits? • Descriptive • what characteristics best describe 3rd y med students? • Explanatory • what factors explain adolescent smoking behavior? • Predictive • what economic factors predict SAT test scores?

16. Survey Research: Advantages • Ecological validity • Access to wide range of participants • Potentially large amounts of data • May be more ethical than experiments

17. Survey Research: Disadvantages • Lack of control - less internal validity • Self-report data only • Potentially low compliance rates

18. Example Survey Research: Lovibond & Gasten (1993) - “Perfectionism and Emotional Symptoms in an Adolescent Sample” • 772 HS students, before & after exams • Measured • perfectionism • depression, anxiety, stress • demographics • Reported relationship between perfectionism & anxiety

19. Building a Survey • Research Question • Constructing Surveys • writing questions • types of responses • measurement error • Levels of Measurement

20. Constructing Questions Aim for these when constructing questions • focus – too specific or too general • brevity - keep as brief as possible • clarity - meaning clear to all respondents • vocabulary - avoid big or ambiguous words, avoid terminology • grammar - simple & correct

21. Constructing Questions Possible sources of bias and error in questions • Inapplicable questions - questions should apply to all respondents • Overdemanding recall - e.g., “How many times have you eaten chicken in the past 10 years?” • Ambiguity - meaning of questions should be clear • Double negatives - “do you not disapprove of the tax reforms?” • Not knowledgeable to answer – ask non-layperson question

22. Constructing Questions • double-barrelled questions - e.g., “Do you think speed limits should be lowered for cars and trucks?” • leading questions - e.g., “don’t you see some danger in the new policy?” • loaded questions “Do you advocate a lower speed limit to save human lives?” Vs “Does traffic safety require a lower speed limit?”

23. Constructing Questions • Question Placement – opening questions, sensitive questions • Assumptions with question wording • Too direct or indirect • Too personal or impersonal

24. Sources of Response Bias • social desirability • acquiescence or yea- and nay-saying - tendency to agree or disagree with everything, use reversed items to control. • “easy out” responses – tendency to answer neutrally instead of working hard for accurate response • self-serving bias - tendency to enhance self • order effects – fatigue, habituation

25. Just World Scale 5 = strongly agree 2 =slightly disagree 4 = moderately agree 1 =moderately disagree 3 = slightly agree 0 =strongly disagree • I’ve found that a person rarely deserves the reputation he has. • Basically, the world is a just place. • People who get ‘lucky breaks’ have usually earned their good fortune. • Careful drivers are just as likely to get hurt in traffic accidents as careless ones. • Students almost always deserve the grades they receive in school.

26. Emotional Expressivity Scale 1 = never true of me 4 = often true of me 2 = rarely true of me 5 = generally true of me 3 = occasionally true of me 6 = always true of me • I think of myself as emotionally expressive. • People think of me as an unemotional person. • I keep my feelings to myself. • I am often considered indifferent by others. • People can read my emotions. • I display my emotions to other people. • I don’t like to let other people see how I’m feeling.

27. Response Format What are the most important problems facing our economy? ______ What are the most important problems facing our economy? (circle response) 1. interest rates 2. unemployment 3. overseas debt 4. consumer confidence 5. other (please specify) • Open questions • (unstructured) • Closed (structured) • Scales (rank, Likert interval scale, etc)

28. Open-ended vs. Close-ended Questions Open-ended • difficult to analyze, subjective analysis • time consuming • rich information • useful for descriptive, exploratory work Close-ended • easier to analyze • efficient • useful for hypothesis testing • important info may be lost forever

29. Ratings Scales • Likert scale • Graphic rating scale • Semantic differential scale • Non-verbal scale • Frequency scale

30. Likert Scale: Evenly spaced categorization of responses Pick a number from the scale to show how much you agree or disagree with each statement: 1 2 3 4 5 strongly disagree neutral agree strongly disagree agree EXAMPLE OF POOR INTERVAL How often you watch baseball games in the summer? 1 2 3 Daily Weekly Never

31. Graphic Rating Scale “How would you rate the movie you just saw?” (Mark with a cross (X)) not very enjoyable very enjoyable ---------------------------------------------------

32. Semantic Differential Scale “Please put a tick in the space on each line below to show your opinion of smoking:” Good ___:___:___:___:___:___:___ Bad Strong ___:___:___:___:___:___:___ Weak Active ___:___:___:___:___:___:___ Passive

33. Non-verbal Scale “Point to the face that shows how you feel about the toy:”

34. Verbal Frequency Scale “Pick a number from the scale to show how often you do the each of the things listed:” 1. Always 2. Often 3. Sometimes 4. Rarely 5. Never

35. Sensitive yet reliable. Too few options - less sensitive. Too many items - less reliable, ceiling & floor effects Depends on question being asked. EXAMPLE “Capital punishment should be reintroduced for serious crimes” 1 = Agree 2 = Disagree VERSUS 1 = Very, Very Strongly Agree 7 = Slightly Disagree 2 = Very Strongly Agree 8 = Disagree 3 = Strongly Agree 9 = Strongly Disagree 4 = Agree 10 = V. Strongly Disagree 5 = Slightly Agree 11 = V V Strongly Disagree 6 = Neutral Which type of scale to use?

36. Measurement Error Measurement error is any deviation from the true value. observed score = true score + measurement error measurement error - random - systematic

37. To avoid measurement error: • use multiple indicators • use sensitive measures • avoid confusing wording in questions/instructions • reduce subject & experimenter effects • accurate data entry and administration

38. Modes of Survey Administration Interview • high demand characteristics • can elicit more information • Privacy concerns!! • Higher response rates • Interviewer distortion possible Questionnaire • lower demand characteristics • information may be less rich • Privacy concerns • Lower response rates email, phone, face-to-face, group

39. Survey Format Checklist • Cover letter or verbal introduction • e.g. Who are you. Credentials. Purpose of survey. Ethical approval. How results will be used. Confidentiality. Complaints. • Instructions • To set “mind frame”, but be aware that for most people “if all else fails, read the instructions” • Group like questions together • Consider order effects, habituation, fatigue, switching between response formats

40. Pre-test & Revise • Pre-test items and ask for feedback • Revise: • items which don’t apply to everybody • redundancy • skewed response items • misinterpreted items • non-completed items • Ordering & layout problems

41. Overview of survey design 1. Define research problem 2. Hypotheses 3. Operationalize constructs (minimize measurement error) 4. Identify target population & sampling frame and technique (minimize sampling error) 5. Administration technique (phone, face) 6. Collect data & analyse 7. Write report.