220 likes | 417 Views
Overview.... . . . . . . . . . . Introduction. The surveys. A look inside. The future . A Survey of Surveys. . Why we started this big task
E N D
1. A Survey of Surveys Karen Decancq, Frank Cowell,
Erik Schokkaert
Katholieke Universiteit Leuven
London School of Economics
May 2009
2. Overview...
3. Purpose History of our group
Barcelona +
Ghent +
Intellectual background
preference elicitation
questionnaire experiments
social justice questionnaires
Why a “Survey of Surveys”?
ESF history
ESS
Role of this presentation
reference document
basis for planning History
Start from Guernsey 2001?
Work together informally – seek funding for more serious stuff
Intellectual
preference elicitation: Alain
questionnaire experiments: Yoram, Frank, Erik, Xavi
social justice questionnaires: Erik, Kurt
Why S-o-S
Explain first ESF grant that ended end 2006
ESF workshops didn’t work, but we worked together anyway
From Barcelona onwards anew core plan
measurement of social mobility and equality of opportunity brings together topics and techniques cited in Intellectual background
This presentation
We need to have a good feel for what others are doing / have done
Particularly in areas outside economicsHistory
Start from Guernsey 2001?
Work together informally – seek funding for more serious stuff
Intellectual
preference elicitation: Alain
questionnaire experiments: Yoram, Frank, Erik, Xavi
social justice questionnaires: Erik, Kurt
Why S-o-S
Explain first ESF grant that ended end 2006
ESF workshops didn’t work, but we worked together anyway
From Barcelona onwards anew core plan
measurement of social mobility and equality of opportunity brings together topics and techniques cited in Intellectual background
This presentation
We need to have a good feel for what others are doing / have done
Particularly in areas outside economics
4. What Surveys? In principle, several types of survey could be relevant
Subjective: Attitudes
social justice
implicit tradeoffs
what is the role of government?
Subjective: Concepts
what is meant by economic equality?
what is meant by equality of opportunity?
what is meant by a more mobile society?
Subjective: Perceptions
what does the income distribution look like?
what is the burden of taxation?
Objective: Facts
income, other resources
personal circumstances
family, environment Subjective: Attitudes
implicit tradeoffs
responsibility versus compensation – where to draw the line?
Subjective: Perceptions
did you yourself have “equal opportunities/chances in life”?
do children in general have “equal opportunities/chances in life?”
Objective: Facts
family, environment: be explicit? E.g. SES parents, childhood environment
In our approach
summarise attitudes + concepts + perceptions as “values”
Subjective: Attitudes
implicit tradeoffs
responsibility versus compensation – where to draw the line?
Subjective: Perceptions
did you yourself have “equal opportunities/chances in life”?
do children in general have “equal opportunities/chances in life?”
Objective: Facts
family, environment: be explicit? E.g. SES parents, childhood environment
In our approach
summarise attitudes + concepts + perceptions as “values”
5. A Way Forward Simple catalogue
what is the purpose and scope of the survey?
how is the survey carried out?
what’s in it for us?
Thematic approach
helps wiifu
helps in comparing the surveys
Find the gaps
partial coverage
new concepts
could we fill them?
Major project development Catalogue
more than just a tour d’horizon
understand how other professionals approach these tasks
self-education if we hope to be part of it
Thematic
we’ve tried to simplify
should tie into interests of our group
Project
one of these surveys – ESS is of special interest to us
more on this later
Catalogue
more than just a tour d’horizon
understand how other professionals approach these tasks
self-education if we hope to be part of it
Thematic
we’ve tried to simplify
should tie into interests of our group
Project
one of these surveys – ESS is of special interest to us
more on this later
6. Overview... -
screen existing European (and global), representative surveys on their content relevant for the topic of equality of opportunity and intergenerational mobility.
- Frank and Erik asked to prepare a note which could be considered the benchmark on which further research should build – e.g. by using the existing results, designing ‘missing’ questions, identifying and targeting ‘missing’ populations and running a ‘new’ questionnaire.
- go through methods and organisation of S-o-S
- For this purpose: go through 6 different surveys-
screen existing European (and global), representative surveys on their content relevant for the topic of equality of opportunity and intergenerational mobility.
- Frank and Erik asked to prepare a note which could be considered the benchmark on which further research should build – e.g. by using the existing results, designing ‘missing’ questions, identifying and targeting ‘missing’ populations and running a ‘new’ questionnaire.
- go through methods and organisation of S-o-S
- For this purpose: go through 6 different surveys
7. Introduction to the S-o-S The European Social Survey (ESS)
The European Values Study (EVS)
The World Values Survey (WVS)
The International Social Survey Programme (ISSP)
The Gallup World Poll
The European Union Statistics on Income and Living Conditions (EU-SILC) Survey quality of ESS is really good (almost superior – e.g. only face-to-face, sampling should be approved, fixed goal of 70% response rate) , but “relatively little” attitudinal info on EOp
That ‘s why also look at other surveys.
Survey quality of ESS is really good (almost superior – e.g. only face-to-face, sampling should be approved, fixed goal of 70% response rate) , but “relatively little” attitudinal info on EOp
That ‘s why also look at other surveys.
8. Introduction to the S-o-S (2) General structure
What, when and where?
Methodology and sampling
How?
Availability
How to get access to data?
What is in it for us? methodology and organisation
sampling = aim is random sampling, but actual method of achieving this requirement will vary considerably between participating countries, depending on their access to sampling sources and other considerations. alternative: quota sampling methodology and organisation
sampling = aim is random sampling, but actual method of achieving this requirement will vary considerably between participating countries, depending on their access to sampling sources and other considerations. alternative: quota sampling
9. The European Social Survey (ESS) What?
How European values and behaviour patterns are distributed, differ within/between countries, change
Cross-sectional and longitudinal survey
When and where?
every two years, from 2002 onwards
core module + rotating module
Methods and sampling
National samples (min 1500 interviews)
Aged 15+
Availability
ESS data website, without restrictions
Cumulative data file Round 1-3
Round 4: by end of September 2009 Social variables
including media use, social and public trust; political interest and participation; socio-political orientations, governance and efficacy; moral, political and social values; social exclusion, national, ethnic and religious allegiances; well-being, health and security; demographics and socio-economics.
Rotating modules:
Round 1 Citizenship, Involvement and Democracy ; Immigration
Round 2 Family, Work & Well-Being Opinions on Health & Care Seeking Economic Morality in Europe: Market Society&Citizenship
Round 3 Personal & Social Well-being ; The Timing of Life
Round 4 Ageism; Welfare attitudes in a changing Europe
Round 5 Work, Family and Well-being Trust in Criminal Justice
Cumulative data file Round 1-3 since October 2008
Survey quality of ESS is really good (almost superior – e.g. only face-to-face, sampling should be approved, fixed goal of 70% response rate) , but “relatively little” attitudinal info on EOp
That ‘s why also look at other surveys.Social variables
including media use, social and public trust; political interest and participation; socio-political orientations, governance and efficacy; moral, political and social values; social exclusion, national, ethnic and religious allegiances; well-being, health and security; demographics and socio-economics.
Rotating modules:
Round 1 Citizenship, Involvement and Democracy ; Immigration
Round 2 Family, Work & Well-Being Opinions on Health & Care Seeking Economic Morality in Europe: Market Society&Citizenship
Round 3 Personal & Social Well-being ; The Timing of Life
Round 4 Ageism; Welfare attitudes in a changing Europe
Round 5 Work, Family and Well-being Trust in Criminal Justice
Cumulative data file Round 1-3 since October 2008
Survey quality of ESS is really good (almost superior – e.g. only face-to-face, sampling should be approved, fixed goal of 70% response rate) , but “relatively little” attitudinal info on EOp
That ‘s why also look at other surveys.
10. The European Values Study (EVS) What?
Investigation of attitudes towards basic human values
Cross-national and longitudinal survey
When and where?
Round 1 1981 14 European countries
Round 2 1990 31 European countries + US, Canada
Round 3 1999 almost all Europe + …
Round 4 2008 46 European countries + …
Methods and sampling
National samples (target 1000 interviews)
Aged 18+
Availability
ZACAT, without restrictions
Data 2008 : available late 2009/early 2010 Basic human values: family, work, religion, politics and society
When: initiated by the European Value Systems Study Group (EVSSG) in the late 1970s.
first round (1981), in 14 European countries (Belgium, Denmark, France, Germany, Great Britain, Iceland, Ireland, Italy, Malta, The Netherlands, Northern Ireland, Norway, Spain and Sweden).
aroused interest in N and S America, the Middle and Far East, Australia, and South Africa
affiliated groups set up to administer the same questionnaire. exchange of data for intercontinental and intercultural comparisons (early start of World Values Studies).
second wave launched in 1990 in all European countries (1981 countries + Austria, Belarus, Bulgaria, Czech Republic, Estonia, Finland, Germany East, Hungary, Latvia, Lithuania, Poland, Portugal, Romania, Russia, Slovak Republic, Slovenia, Switzerland), as well as the US and Canada.
ten years later (1999), the third EVS wave was conducted in almost all European countries (1990 countries with the exception of Germany, Norway and Switzerland + Croatia, Greece, Luxembourg, Turkey, Ukraine). Alllowed investigation of causes and consequences of dynamics of value change.
In 2008, the fourth wave. The fieldwork covered no less than 46 European countries (1999 countries + Albania, Armenia, Azerbaijan, Bosnia-Herzegovina, Cyprus, Germany, Kosovo, Moldova, Montenegro, Norway, Serbia and Switzerland).
(FYI: ZACAT is a social science data portal allowing you to search for, browse, analyse and download social science survey data. The studies in this catalogue present a small selection of the studies available at the GESIS Data Archive - however the ZACAT holdings are extended on a regular base. Beside documentation on study level (e.g. abstract, related publications), all studies come along with documentation of full question and answer texts on variable level. Moreover, for some of the collections documentation on variable level is offered in two or more languages. Among others: EVS, ISSP, Eurobarometer)
Availability: first three waves: format and contents of the surveys have changed. Variable names are different between surveys, questions have been removed or adjusted and new topics have been added. Online document where you can view variables that are included in the 1981-, 1990- and 1999-datafiles. The document gives an overview of the existing variables and it also shows which of the variables in the three surveys are comparable in time.Basic human values: family, work, religion, politics and society
When: initiated by the European Value Systems Study Group (EVSSG) in the late 1970s.
first round (1981), in 14 European countries (Belgium, Denmark, France, Germany, Great Britain, Iceland, Ireland, Italy, Malta, The Netherlands, Northern Ireland, Norway, Spain and Sweden).
aroused interest in N and S America, the Middle and Far East, Australia, and South Africa
affiliated groups set up to administer the same questionnaire. exchange of data for intercontinental and intercultural comparisons (early start of World Values Studies).
second wave launched in 1990 in all European countries (1981 countries + Austria, Belarus, Bulgaria, Czech Republic, Estonia, Finland, Germany East, Hungary, Latvia, Lithuania, Poland, Portugal, Romania, Russia, Slovak Republic, Slovenia, Switzerland), as well as the US and Canada.
ten years later (1999), the third EVS wave was conducted in almost all European countries (1990 countries with the exception of Germany, Norway and Switzerland + Croatia, Greece, Luxembourg, Turkey, Ukraine). Alllowed investigation of causes and consequences of dynamics of value change.
In 2008, the fourth wave. The fieldwork covered no less than 46 European countries (1999 countries + Albania, Armenia, Azerbaijan, Bosnia-Herzegovina, Cyprus, Germany, Kosovo, Moldova, Montenegro, Norway, Serbia and Switzerland).
(FYI: ZACAT is a social science data portal allowing you to search for, browse, analyse and download social science survey data. The studies in this catalogue present a small selection of the studies available at the GESIS Data Archive - however the ZACAT holdings are extended on a regular base. Beside documentation on study level (e.g. abstract, related publications), all studies come along with documentation of full question and answer texts on variable level. Moreover, for some of the collections documentation on variable level is offered in two or more languages. Among others: EVS, ISSP, Eurobarometer)
Availability: first three waves: format and contents of the surveys have changed. Variable names are different between surveys, questions have been removed or adjusted and new topics have been added. Online document where you can view variables that are included in the 1981-, 1990- and 1999-datafiles. The document gives an overview of the existing variables and it also shows which of the variables in the three surveys are comparable in time.
11. The World Values Survey (WVS) What?
Worldwide investigation of socio-cultural and political change
Cross-national and longitudinal
Together with EVS: 88% of world population
When and where?
5 waves from 1981 to 2008
up to 80 countries
Methods and sampling
Face-to-face interviews
Sample: 1000 to 3500 interviewees, aged 18-85
Availability
WVS website, without restrictions
Also four wave aggregate of WVS/EVS When and where?
The World Values Survey first emerged out of the European Values Study (EVS) in 1981, when the methods of a successful European study were extended outside Europe. The 1981 study nevertheless covered only 20 countries worldwide. The survey was repeated after an interval of about 10 years in the second of what came to be termed "waves". One of the aims of the project came to be the longitudinal (as well as cross-cultural) measurement of variation of values. Further waves followed the second wave at intervals of approximately 5 years.
Due to the European origin of the project, the early waves of the WVS were eurocentric in emphasis, with especially weak representation in Africa and South-East Asia. To expand, the WVS adopted a strongly decentralised structure. Suitable academic representatives from new countries were free to join in a quasi-democratic network. Joining meant that new representatives had to conduct the predefined survey in their own country with at least 1000 interviewees. They could then exchange their data with the WVS in return for the data from the rest of the project. Funding was primarily local, with representatives in each country funding their own part of the project. In this way the WVS grew out of its eurocentric origins to embrace 42 countries in the 2nd wave, 52 in the 3rd wave, 67 in the 4th wave and 54 in the 5th wave.
When and where?
The World Values Survey first emerged out of the European Values Study (EVS) in 1981, when the methods of a successful European study were extended outside Europe. The 1981 study nevertheless covered only 20 countries worldwide. The survey was repeated after an interval of about 10 years in the second of what came to be termed "waves". One of the aims of the project came to be the longitudinal (as well as cross-cultural) measurement of variation of values. Further waves followed the second wave at intervals of approximately 5 years.
Due to the European origin of the project, the early waves of the WVS were eurocentric in emphasis, with especially weak representation in Africa and South-East Asia. To expand, the WVS adopted a strongly decentralised structure. Suitable academic representatives from new countries were free to join in a quasi-democratic network. Joining meant that new representatives had to conduct the predefined survey in their own country with at least 1000 interviewees. They could then exchange their data with the WVS in return for the data from the rest of the project. Funding was primarily local, with representatives in each country funding their own part of the project. In this way the WVS grew out of its eurocentric origins to embrace 42 countries in the 2nd wave, 52 in the 3rd wave, 67 in the 4th wave and 54 in the 5th wave.
12. The International Social Survey Programme (ISSP) What, when and where?
Attitudes towards social issues
Supplement to regular national surveys
Topical modules on attitudes towards social issues
Social inequality 1987, ‘92, ‘99, 2009
Role of government 1985, ‘90, ‘96, 2006
Family and changing gender roles 1988, ‘94, 2002
Participating countries vary for each module (recently about 40 countries)
Methods and sampling
Minimum 1000 achieved sample
Mixture of face-to-face and self-completion method
Often administered as part of another survey
Availability
ZACAT, without restrictions What?
questions about general attitudes toward various social issues such as the legal system, sex and the economy. Special topics have included the role of government, social inequality, social support, family and gender issues, work orientation, the impact of religious background, behaviour, the environment, beliefs on social and political preferences, and national identity. What?
questions about general attitudes toward various social issues such as the legal system, sex and the economy. Special topics have included the role of government, social inequality, social support, family and gender issues, work orientation, the impact of religious background, behaviour, the environment, beliefs on social and political preferences, and national identity.
13. Gallup World Poll What?
Reactions to events that have changed the world
Cross-sectional and longitudinal survey
Organised into performance indices
e.g. well-being index
When and where?
In more than 150 countries, representing 95% of world’s population
Methods and sampling
face-to-face or by telephone
aged 15+
min 1000 interviews per country
Availability
Gallup WorldView
Full Respondent-Level Data
info on pricing by mail
14. EU-SILC What?
Facts on income, housing, labour, health, demography, education
= multidimensional approach of social exclusion
cross-sectional and longitudinal
replaces ECHP (1994-2001)
When and where?
from 2003 onwards
member states of European Union
organised into primary and secondary target variables
Methods and sample
cross-section: minimal 121 000 households, 250 000 individuals
longitudinal: minimal 187 000 individuals
aged 16+
Availability
micro data for researchers via CD-ROM The European Union Statistics on Income and Living Conditions (EU-SILC)
Over an eight year period (from 1994 to 2001), the ECHP (European Community Household Panel), ran in 14 of the then 15 Member States (with the exception of Sweden), and served as the source for many of the commonly agreed social inclusion indicators for this period.From the beginning, the ECHP project suffered from some quality problems, mainly, incomplete geographical coverage, reliability and timeliness. It is with the aims of solving the ECHP technical problems, of conforming to the internationally agreed definition of income and of extending the data collection to the enlarged EU (and beyond) that the decision was taken to stop the ECHP and launch EU-SILC.The European Union Statistics on Income and Living Conditions (EU-SILC)
Over an eight year period (from 1994 to 2001), the ECHP (European Community Household Panel), ran in 14 of the then 15 Member States (with the exception of Sweden), and served as the source for many of the commonly agreed social inclusion indicators for this period.
15. Overview...
16. Content: Facts Facts about interviewee
Facts about household
Facts about father/mother
Facts about childhood
In grey: variables in pretestIn grey: variables in pretest
17. Equal opportunity in an intergenerational context
Equal opportunity in terms of resources
Freedom of choice, control, responsibility, effort, luck & equality
Role of government
Inequality versus mobility
“Acceptable reasons” for income differences Content: Values In grey = similar question in pretest
Additional remark of Erik:
Emphasize that values are attitudes + concepts + perceptionsIn grey = similar question in pretest
Additional remark of Erik:
Emphasize that values are attitudes + concepts + perceptions
18. Gaps Resources and equality of opportunity
both facts and values
e.g. “how treated by schooling system and health care?”
Information on parents (and grandparents?)
e.g. income
Childhood environment of the respondents
e.g. health, “cultural” environment, neighbourhood, details about educational experiences
Meaning of mobility and EOp
e.g. Ex ante versus Ex post
Representative information on responsibility vs compensation
effort and talent: included ; education and health: not included
Responsibility of children or responsibility of parents?
how do they shift over time?
19. Overview...
20. The Marseille Pretest Relation to existing surveys
Some “new” questions…
Mobility
“Mobile society is a society where …”
“I want to live in a more mobile society”
“I want my children to live in a more mobile society”
High revenues
Acceptable that sport stars, singers, advocates have high revenues due to talent and effort?
Socially unacceptable that children of rich parents have …
What is important for EOp of children?
General case, similar to own family, rich family, poor family We need to consider how to exploit the information on “gaps”
Erik, Frank + others suggested the way they could be formulated into specific questions
On this slide only the topics that were NOT in the existing surveys, but that were in the Marseille pretest
Alain will talk about the results
But we may want to talk about how these ideas could be developed for he future We need to consider how to exploit the information on “gaps”
Erik, Frank + others suggested the way they could be formulated into specific questions
On this slide only the topics that were NOT in the existing surveys, but that were in the Marseille pretest
Alain will talk about the results
But we may want to talk about how these ideas could be developed for he future
21. ESS for us? Applying for rotating modules in ESS
Call for proposals placed in Official Journal of the European Union (OJEU)
Call for Round 6 later in 2009 or during 2010
likely to be both for both ‘new’ and ‘repeated’ modules
documents from Round 5 call are available on ESS website, but will change ! lots of experience working with student samples
do we want to pursue a representative-sample study?
if so, ESS may be the most promising
important to understand how it works lots of experience working with student samples
do we want to pursue a representative-sample study?
if so, ESS may be the most promising
important to understand how it works
22. What Next? Suggestions for further work
Allocation of specific topics to different group members
A very careful study of the Survey-of-Surveys
Theoretical question formulated clearly
combination older question/new one
Proposal for ESS
other “survey providers”? Perhaps connect
(1) Allocation of specific topics
(2) Study of Survey of surveys
Homework approach has worked well for this meeting – and the future?
Other survey providers include Gallup who are keen to have interaction with academics
Perhaps connect
(1) Allocation of specific topics
(2) Study of Survey of surveys
Homework approach has worked well for this meeting – and the future?
Other survey providers include Gallup who are keen to have interaction with academics