390 likes | 401 Views
This study aims to monitor and interpret public attitudes and values within Europe, investigating how they interact with changing institutions. It provides data on beliefs, attitudes, and values for scientific and policy making purposes, with a focus on measuring attitude change in a changing Europe. The study collects data using face-to-face surveys in over 30 European countries and follows the highest standards of survey measurement.
E N D
From face-to-face to web in a cross-nationalstudy Ineke Stoop SCP/CST
ESS • Academic survey in 30+ European countries • Attitudes, values and beliefs • Biennial, start in 2002 • 1+ hour, face-to-face, effective sample size ~ 1500 • Funders • National Science organisations • European Commission, Framework Programmes • European Science Foundation • Management model • Central coordination • National implementation
Aims ESS To monitor and interpret public attitudes and values within Europe and to investigate how they interact with Europe’s changing institutions Provide data on beliefs, attitudes and values for scientific and policy making purposes Measure attitude change in a changing Europe To advance and consolidate improved methods of cross-national survey measurement in Europe and beyond Collect data according to highest standards Generate methodological research Develop and disseminate new best practices Develop and improve social survey research infrastructure in Europe
Content questionnaires ESS Everyround Roundspecific R1 Immigration Citizenship, involvement and democracy R2 Family, work and well-being Opinions on health and care- seeking Economic Morality R3 Personal & Social Well-being The Timing of Life R4 Ageism Welfare attitudes R5 Family, work and well-being in times of economic recession Trust in police and courts R6 Personal & Social Well-being Understanding, evaluation democracy • Socio-demo-graphic items • household composition • education • housing • occupation • social structure • Attitudes and behaviour patterns • religious affiliation and identity • ethnic and national identity • political and social trust • party affiliation and voting behaviour • media consumption • value orientations • social exclusion
Access ESS Round 1-5 available Fully documented Free access Tables and graphics online, or download data Training package for young researchers Everything available All deviations documented
Methods: high quality • Strictly random sampling • No substitution, no proxy • Population: all inhabitantsaged 15+ • High target response rates • 70% target response rate, max. 3% noncontact • Face-to-face fieldwork • Gold standard, high response rates, universally applicable • Increasingly expensive • Rare in some countries • Pilots, pretesting and fieldwork monitoring • Analysis of cross-national differences • Nonresponse bias, measurement error, sampling design effects
Methods: central coordination Central specifications for fieldwork Close monitoring national fieldwork (in advance and during fieldwork) Sampling design Translation procedures Guidelines response enhancement Guidelines interviewer training Contact forms, response calculation Context, events
Can we comparecountries? Attitudes, values, beliefs Survey results Survey design, methodology, implementation
Harmonisation strategies (Körner and Meyer, 2005) Harmonised concept Harmonised concept Harmonised concept Measurement procedure Measurement procedure Measurement procedure National survey/ Specific concept National survey/ Specific concept National survey/ Specific concept Input harmonisation Ex-ante output harmonisation Ex-post output harmonisation
Output harmonisation • Ex post output harmonisation • Use existing sources • Ex ante output harmonisation • Set up new survey (or develop new instrument) • Output harmonisation • Best national quality • Or, national survey tradition • But, what about optimal comparability
Output harmonisation: problems • Interview mode • Context question • Lay-out questionnaire • Question text • Introductory text • Answer scales • Handling don’t know • Size of boxes • Use of illustrations
Input harmonisation • Input harmonisation • Design new survey • Optimal comparability • Or, are identical methods and instruments really equivalent in different countries? • But, what about optimal quality
From face-to-face to web in the ESS • Mixed mode research and experiments • Different survey modes over time • Different survey modes across countries • Mixed mode within countries: different composition over time • Which modes: • Face-to-face • Telephone • Web
ESS studies • Martin, Peter (2011) A Good Mix? Mixed Mode Data Collection and Cross-national Surveys. ASK Research & Methods, Vol. 20 (1, 2011) 5–26. • Martin, Peter (2011) What makes a good mix? Chances and Challenges of Mixed Mode Data Collection in the European Social Survey. CCSS Working Paper No. 02. • Martin, Peter, and Lynn, Peter (2011) The Effects of Mixed Mode Survey Designs on Simple and Complex Analyses. CCSS Working Paper No. 02. • Roberts, Caroline (2007) Mixing Modes of Data Collection in Surveys: A Methodological Review. ESRC National Centre for Research Methods. NCRM Methods Review Papers, NCRM/008.
From face-to-face to web in the ESS • Coverage • Sampling • Nonresponse • Questionnaire design • Role interviewers • Measurement • Analysis • Cross-national comparison
Internet access and broadband connections in households, EU27 (%)
Individuals who used the internet at least once a week, 2011 (%)
Individuals who used the internet on average at least once a week, by sex, age and education, EU27, 2011 (%)
Sampling principles • Identical population definition • Complete coverage • No upper age limits • (Institutional population) • (Non-native language speakers) • Probability sampling • Effective sample size • Optimal sampling design in each country • Address, household, individual • Address and household: selection of target respondent in the field
ESS R2 sampling frames (Häder and Lynn) Simple random sampling, or simple stratified sampling (no design effect)
Comparison sampling frames • Achim Koch (GESIS) • In household and address samples: selection of respondent by interviewer • Leads to overrepresentation of women in household/address samples • Related to higher response rates • Close control on who answers the questionnaire in a household even more problematic in web survey
ESS 1 – 4:Sample ofhhs/addresses: Bias X response rate r = .44
Nonresponse ESS • Response targets • 70% response rate • 3% noncontact rate • Standards in • Response rate calculation • Contact form • Nonresponse analysis
Response efforts and monitoring • Guidelines response enhancement • Incentives/Brochure • Refusal conversion • Call schedule requirements • Minimum number of calls • Timing of calls (evening, weekend) • Contact form • Interviewer observations • Call records
Nonresponse • Response rate highest for face-to-face • Functional illiteracy (differs across countries) • Nonresponse analysis • What happens in web • Noncontact, refusal, not able • Reasons for refusal • Interviewer judgment • Refusal conversion
Questionnaire design • Questionnaire length • Split questionnaire • Matrix design • Over time • Question design • Unimode • Developed for face-to-face
Role interviewers • Select respondents (+) • Obtain cooperation (+) • Conduct interview • Explain questions (-) • Explain process (+) • Sensitive questions (+/-) • Validity answers (+) • Social desirability (-)
Measurement • Mode effects • Part of analysis • No simple adjustment • Mixed modes: • Selection effects and mode effects • Always comparison face-to-face • Mode in analyses
Cross-national comparison • Comparison over time • Mode change or time change • Comparison across countries • Mode difference or cultural difference • Mixed modes within countries • Selection effect or mode effect • Mode distribution over time • Feasibility of web for ESS (length, type of questions)