Just how comparative are comparative statistics?

1. Just how comparative are comparative statistics? Roger Jowell Centre for Comparative Social Surveys City University LLAKES International Conference

2. Why bother with comparative social statistics? Discovering differences and similarities “Comparative sociology is not a particular branch of sociology. It is sociology itself.” Durkheim Understanding one’s own society Critical for cross-national governance Helps challenge national stereotypes But comparisons ideally needed both between countries AND over time

3. Obstacles to cross-national social measurement Cultural incompatibilities The language barrier Breaching the ‘principle of equivalence’ Temptation of league tables Clash between standards and consistency Differences in methodological capacity Differences in methodological habits

4. Overcoming the obstacles – at least partially Ensuring equivalence of methods in all countries: • Sampling • Mode of data collection • Language equivalence • Concept equivalence • Meticulous documentation • Consultative design, not ‘safari’ method • Contextual variables

5. A case history - the European Social Survey Multinational time series, started 2001 34 countries so far Contributes to scholarship and governance Large training component and potential Several substantive and methodological innovations Widespread usage

6. Five main aims To chart and explain changes in Europe’s social, political and moral climate To achieve and spread high standards of rigour in comparative social measurement To establish new indicators of societal well-being to stand alongside existing factual and behavioural indicators To facilitate quantitative monitoring of value change by academics, policymakers, businesses and the public To create and maintain a new contextual data repository

7. ESS Countries Iceland Luxembourg Bulgaria Germany Netherlands Poland Slovenia Hungary Czech Republic Austria Estonia Ukraine Belgium Latvia Denmark Switzerland Russia Sweden Turkey Headquarters London, UK Israel Lithuania France Slovakia Italy Portugal Spain Romania Ireland Croatia Norway Finland Greece UK Cyprus

8. ESS Countries by Round

9. Funding story to date Initiated and seed-funded by European Science Foundation Then core-funded for five biennial rounds (to date) by European Commission National costs of each round met by national academic funding councils Over 30 separate funding decisions each round Now selected as a prospective ‘ESFRI’ Research Infrastructure with prospect of long-term funding

10. National, ethnic, religious ID Health and welfare issues Life course perceptions Ageism Work and family life Education and occupation Financial circumstances Household circumstances Demographic composition Question clusters – some core, some rotating Trust in institutions Citizen engagement Socio-political values Immigration Moral & social attitudes Quality of life Crime and security Value orientations Perceptions of criminal justice

11. Outreach 32,000 registered data users to date (2500+ in UK) On-line bibliography of publications based on ESS contains: 236 journal articles, 36 books and 90 chapters so far Data increasingly deployed in policy debates Training courses heavily over-subscribed Influence on comparative methods well beyond Europe

12. A few summary findings

13. Education and Political Interest

14. Four Dimensions of Institutional Trust… Trust in the police varies but is high…

15. Four Dimensions of Institutional Trust… Trust in the legal system is lower….

16. Four Dimensions of Institutional Trust… Trust in national parliaments is lower still…

17. Four Dimensions of Institutional Trust… Trust in politicians is the lowest of all.

18. Attitudes to migration Surges of xenophobia associated with economic downturns New dangers of recession But education matters More education, less xenophobia, greater sympathy towards cultural diversity Educated are more accepting of all newcomers, even of potential labour market competitors

19. 10 Laws of comparative research

20. Law 1 Don’t confuse respect for cultural differences with tolerance of methodological anarchy

21. Law 2 Never design questions or interpret data about a country one knows little or nothing about

22. Law 3 Confine cross-national studies to the smallest number of nations compatible with the study’s intellectual needs

23. Law 4 Pay as much attention to collecting aggregate-level background information about each country as to individual-level variables

24. Law 5 Always be at least as absorbed by the limitations of the data as about their explanatory power

25. Law 6 Assume initially that any major ‘new’ cross-national variation one discovers is an artefact

26. Law 7 Resist the temptation to produce ‘gee-whiz’ league tables containing every nation in every analysis

27. Law 8 Undertake collective, study-specific, multi-national development work and pre-testing

28. Law 9 Routinely include methodological experiments in cross-national studies

29. Law 10 Ensure that cross-national datasets are accompanied by detailed methodological reports about procedures and outcomes in each nation