1 / 29

Understanding Society Conference 25 July 2013

Question ordering effects on the reporting of fertility intentions and close social networks. Understanding Society Conference 25 July 2013 Paul Mathews Knowledge , Analysis and Intelligence Directorate, HM Revenue and Customs Institute of Social and Economic Research, University of Essex.

kendra
Download Presentation

Understanding Society Conference 25 July 2013

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Question ordering effects on the reporting of fertility intentions and close social networks Understanding Society Conference 25 July 2013 Paul MathewsKnowledge, Analysis and Intelligence Directorate, HM Revenue and Customs Institute of Social and Economic Research, University of Essex

  2. Question ordering - Context Effects • Change in the answers to a survey questionnaire as a function of the previous items in the questionnaire’ Tourangeau et al, 2003 • Examples • Context Effects • Vodka or beer questions influences rating to how ‘Germanic’ is wine drinking? (Schwarz, Munkel and Hippler, 1990) • Life Satisfaction preceding Marriage Satisfaction r = 0.32, Marriage satisfaction preceding Life Satisfaction r = 0.67 (Schwarz, Strack and Mai 1991) • Frequency of Context Effects • General Social Survey (US) batteries of questions rotated. Only 4% of questions effected by placement (Smith, 1988) • Needs to be a conceptual link

  3. Question ordering - Context Effects • Question priming bias as domain sampling • Particularly in multipurpose longitudinal research (Time series - Change over time? Changes in preceding questions?) • Plausible risk

  4. Why are fertility intentions important?

  5. “The changing face of London: A baby boom is sending the city’s planners back to the drawing board” • The Economist 28th Jan 2012 “By 2015-16 greater London will need around 70,000 more school places”

  6. Measurement problems… • Uncertainty / ambivalence • Context dependent… • Preferences change over time • Age, ageing, life course, cohort, period • Is there a ‘correct’ age to measure FP? • Experience of children • Partnership and partner’s preferences • Competing preferences • economic, cultural, leisure etc… • Because fertility preferences are so context dependent, then will the context in the questionnaire matter i.e. preceding questions?

  7. Millennium Cohort Study – Wave 1 “How long did the labour last?” “Which, if any, of the following types of pain relief did you have at any time during labour?” Before asking “Do you plan to have any more children?”

  8. Social networks • Numerous concepts and operationalisations • Flows through social networks • Social capital • Strength of weak ties • Relatedness • At risk of context effects? E.g. prime a domain such as ‘work’ or ‘family’… does this influence who is ‘in’ your social network

  9. My empirical work

  10. Mortality experiments Randomised (systematically identical) groups. • Treatments: priming questions then fertility questions • Controls: fertility questions then priming questions Adult (own) mortality priming questions • 11 Questions • “What age do you expect to be when you die?” • Data collected 2006 and 2008-09 • Published - Mathews and Sear 2008 • Students internet experiment • Results: Significant increase in MALE ideal numbers of children. No effect for females

  11. Why? Not mutually exclusive… • Fatigue? • Negative mood? • Old age support (in adult prime)? • First item in battery of fertility preferences? (DHS ideal question) • Own mortality is a ‘shock’ to non-decision decision? (Competing preferences, cultural output and sociological modernity) • Social Psychological - Terror Management Theory (TMT) social immortality? • Evolutionary biology – Life History Theory (perceived risky environment should alter reproductive strategy)? • Sheer chance?! • Replication

  12. Innovation Panel experiment • Waves 4 and 5 of Innovation Panel sub sample of 1,500 households - NOT STUDENTS! • Randomisation at household level • Controls • Wave 4: Experiment after mental wellbeing • “I've been able to make up my own mind about things” • 5 point scale: [All of the time – None of the time] • Wave 5: Experiment after GHQ • “Have you recently been feeling reasonably happy, all things considered?” • 4point scale 1 More so than usual 4 Much less than usual

  13. Two question ordering ‘treatments’ • Fertility Intentions: • “Do you think you will have any (more) children?” • [1 Yes, 2 Self / partner currently pregnant, 3 No] • if the answer is yes “How many (more) children do you think you will have?” • Close social network (i.e. 3 closest friends) • ‘Please choose the three people you consider to be your closest friends... They should not include people who live with you but they can include relatives’ • Sex, Age, relatedness, frequency of contact, how far away they live etc • ‘Is this friend a relative?’ • [ 1 Yes, 2 No]

  14. Descriptive statistics • Observations 696 • Wave 4 N=409, Wave 5 N=287 • 223 individuals measured twice (27 changed their minds on wanting children) • Background demographics remain very similar across waves - Male 60%, Age mean 37.5 (SD 13.4) median 39 (split dummies in model), Parents 48%, Employed 72% (11% full time students), Married 45%, Lives with a parent 22%, sibling 14%.

  15. Fertility intentions 27 close social network questions (Nine questions for three friends) wave 4 ‘Make mind’ or wave 5 ‘happiness’ Fertility intentions questions (No social network questions)

  16. Social network 1 or 2 fertility intentions questions Close social network questions wave 4 ‘Make mind’ or wave 5 ‘happiness’ (No fertility intentions questions)

  17. Results – Fertility intentions

  18. Results – Fertility intentions

  19. Results – Fertility intentions

  20. Results – Social network

  21. Conclusions • Fertility intentions at risk of preceding questions • Plausible risk... • Little evidence relatedness (or any other characteristics) of their close social network at risk of preceding questions • Important to construct and read questionnaires as a whole • Repeated measures: Replication, replication, replication

  22. Acknowledgements Participants in all studies Maria Iacovou, University of Essex Rebecca Sear, London School of Hygiene and Tropical Medicine Ernestina Coast London School of Economics and Political Science UK Economic and Research Council for funding UKHLS Methodological Advisory Committee for accepting proposal ISER and HMRC secondment

  23. *Advert* - HM Revenue & Customs Datalab • Compliance • Corporation tax • Self assessment • Value added tax • Stamp duty land tax • Trade statistics • Tax credits • Tobacco • Variable names and descriptions are available on our website: • www.hmrc.gov.uk/datalab/data.htm

  24. Conclusions • Fertility intentions at risk of preceding questions • Plausible risk... • Little evidence relatedness (or any other characteristics) of their close social network at risk of preceding questions • Important to construct and read questionnaires as a whole • Repeated measures: Replication, replication, replication

More Related