European Statistical System and European Social Survey

1. European Statistical System and European Social Survey Input and output harmonisation Ineke Stoop (SCP)

2. Harmonisation strategies (Körner and Meyer, 2005) Harmonised concept Harmonised concept Harmonised concept Measurement procedure Measurement procedure Measurement procedure National survey/ Specific concept National survey/ Specific concept National survey/ Specific concept Input harmonisation Ex-ante output harmonisation Ex-post output harmonisation Q2010, Helsinki, Harmonisation models

3. Type of harmonisation Ex post output harmonisation Use existing sources Ex ante output harmonisation Set up new survey (or develop new instrument) Output harmonisation Best national quality Or, national survey tradition But, what about optimal comparability Input harmonisation Design new survey Optimal comparability Or, are identical methods and instruments really equivalent in different countries? But, what about optimal quality Q2010, Helsinki, Harmonisation models

4. European Statistical System: output harmonisation • Registers orsurveys? • Comparisonwithincountries • Target population • Survey mode • Questiontext (examples, scales) • Fieldwork • Proxy, substitution • Nonresponse • Efforts, rate, bias Q2010, Helsinki, Harmonisation models

5. European Social Survey • Attitudes, values and beliefs • Bi-annual • 2002, 2004, 2006, 2008 • Small sample size: effective size 1500 • 30+ countries • EU, Switzerland, Norway, Russia, Ukraine, Turkey, Israel Q2010, Helsinki, Harmonisation models

6. Aims ESS • To monitor and interpret public attitudes and values within Europe and to investigate how they interact with Europe’s changing institutions • Provide data on beliefs, attitudes and values for scientific and policy making purposes • Measure attitude change in a changing Europe Q2010, Helsinki, Harmonisation models

7. Substance: core modules • Socio-demo-graphic items • household composition • education • housing • Occupation • Attitudes and behaviour patterns • religious affiliation and identity • ethnic and national identity • political trust • party affiliation and voting behaviour • media consumption • value orientations • social exclusion Q2010, Helsinki, Harmonisation models

8. Rotating modules: scientific community • R1 • Immigration • Citizenship, involvement and democracy • R2 • Family, work and well-being • Opinions on health and care-seeking • Economic Morality in Europe: Market Society & Citizenship • R3 • Personal & Social Well-being: Creating indicators for a flourishing Europe • The Timing of Life: The organisation of the life course in Europe • R4 • Experiences and Expressions of Ageism • Welfare attitudes in a changing Europe • R5 • Family, work and well-being: implications of economic recession • Trust in criminal justice and the police Q2010, Helsinki, Harmonisation models

9. Can this be measured by output harmonisation • Second aim ESS • To advance and consolidate improved methods of cross-national survey measurement in Europe and beyond • Collect data according to highest standards • Generate methodological research • Develop and disseminate new best practices • Develop and improve social survey research infrastructure in Europe • Input harmonisation Q2010, Helsinki, Harmonisation models

10. Harmonisation model Central coordination National implementation Survey organisation Stats, uni, market Interviewer training Sampling design Call schedule Response enhancement strategies • Sampling protocol • Strictly random, no proxy, nosubstitution • Source questionnaire • Translation procedures • Fieldwork specifications • Interview mode (f2f) • Contracting checklist • Fieldwork monitoring • Guidelines and protocols Q2010, Helsinki, Harmonisation models

11. Input harmonisation • Somethingscannotbeharmonised • Language • Sampling frames • Response rates • Level youaimfor • Lowest • Highest • Average • National best Q2010, Helsinki, Harmonisation models

12. Coverage Random sample, butsomegroups are excluded Elderly in institutions Military, student halls, hostels Non-native speakers Not in population register Opt-out Realitydiffersacrosscountries Q2010, Helsinki, Harmonisation models

13. Nonresponse • Response rate • Response efforts • Required/made • Response composition • Noncontact/refusal • Nonresponse bias • Interest in politics • Trust in government • Voluntarywork Q2010, Helsinki, Harmonisation models

14. Response rates ESS Q2010, Helsinki, Harmonisation models

15. Questionnaire design • Ask the same question • Ensure that various concepts are actually represented as precisely as possible • Reliability and validity tests (SQP) • Cognitive interviews • Translation from source language into 1 target language for two-country pilot • Two-nation pilot • Comments national coordinators • Final source questionnaire and annotations • Translation and documentation • Verification • Small pilots in every participating country Q2010, Helsinki, Harmonisation models

16. No back-translation in ESS • Back translation • It’s raining cats and dogs • Have you had your tea yet? • Geniessen in vollen Zügen • Dear undertakers • Perfect back translation does not guarantee a good question in the target language • Good help in finding obvious errors (wealthy/healthy) Q2010, Helsinki, Harmonisation models

17. TRAPD • Translation • More than 1 translator • What is a good questionnaire translator • Training of translators • Review • Discuss and compare • Adjudicate • Decide on 1 version • Pretest • Documentation • Also of review process • Verify Q2010, Helsinki, Harmonisation models

18. Survey mode • Face-to-face • Crucialrole of interviewer • Let someone in house • Interviewer can have an effect onanswers • Usuallyhigher response rates • Long fieldwork period • Expensive • Mail • Lesssociallydesirable • Lower response rates • Whoanswers the questionnaire? • Telephone • Crucialrole of interviewer • Interviewer can have an effect onanswers • Lower response rate • No landlinephone, ex-directory • Short interviews • Internet • Lesssociallydesirable • Lower response rates • Whoanswers the questionnaire? Q2010, Helsinki, Harmonisation models

19. Survey modes Face-to-faceonly Applicablein every country? Do modes mean the same in every country? Q2010, Helsinki, Harmonisation models

20. Fieldwork • Different organisations • House effects • Interviewers • Training, remunerations • Tradition • National survey culture Q2010, Helsinki, Harmonisation models

21. In some cases input harmonisation is the onlyoption Butyou have to beaware of cross-nationaldifferences

22. Thankyouforyourattention Q2010, Helsinki, Harmonisation models