A Survey of Surveys

1. A Survey of Surveys Karen Decancq, Frank Cowell, Erik Schokkaert Katholieke Universiteit Leuven London School of Economics May 2009

2. Overview...

3. Purpose History of our group Barcelona + Ghent + Intellectual background preference elicitation questionnaire experiments social justice questionnaires Why a �Survey of Surveys�? ESF history ESS Role of this presentation reference document basis for planning History Start from Guernsey 2001? Work together informally � seek funding for more serious stuff Intellectual preference elicitation: Alain questionnaire experiments: Yoram, Frank, Erik, Xavi social justice questionnaires: Erik, Kurt Why S-o-S Explain first ESF grant that ended end 2006 ESF workshops didn�t work, but we worked together anyway From Barcelona onwards anew core plan measurement of social mobility and equality of opportunity brings together topics and techniques cited in Intellectual background This presentation We need to have a good feel for what others are doing / have done Particularly in areas outside economicsHistory Start from Guernsey 2001? Work together informally � seek funding for more serious stuff Intellectual preference elicitation: Alain questionnaire experiments: Yoram, Frank, Erik, Xavi social justice questionnaires: Erik, Kurt Why S-o-S Explain first ESF grant that ended end 2006 ESF workshops didn�t work, but we worked together anyway From Barcelona onwards anew core plan measurement of social mobility and equality of opportunity brings together topics and techniques cited in Intellectual background This presentation We need to have a good feel for what others are doing / have done Particularly in areas outside economics

4. What Surveys? In principle, several types of survey could be relevant Subjective: Attitudes social justice implicit tradeoffs what is the role of government? Subjective: Concepts what is meant by economic equality? what is meant by equality of opportunity? what is meant by a more mobile society? Subjective: Perceptions what does the income distribution look like? what is the burden of taxation? Objective: Facts income, other resources personal circumstances family, environment Subjective: Attitudes implicit tradeoffs responsibility versus compensation � where to draw the line? Subjective: Perceptions did you yourself have �equal opportunities/chances in life�? do children in general have �equal opportunities/chances in life?� Objective: Facts family, environment: be explicit? E.g. SES parents, childhood environment In our approach summarise attitudes + concepts + perceptions as �values� Subjective: Attitudes implicit tradeoffs responsibility versus compensation � where to draw the line? Subjective: Perceptions did you yourself have �equal opportunities/chances in life�? do children in general have �equal opportunities/chances in life?� Objective: Facts family, environment: be explicit? E.g. SES parents, childhood environment In our approach summarise attitudes + concepts + perceptions as �values�

5. A Way Forward Simple catalogue what is the purpose and scope of the survey? how is the survey carried out? what�s in it for us? Thematic approach helps wiifu helps in comparing the surveys Find the gaps partial coverage new concepts could we fill them? Major project development Catalogue more than just a tour d�horizon understand how other professionals approach these tasks self-education if we hope to be part of it Thematic we�ve tried to simplify should tie into interests of our group Project one of these surveys � ESS is of special interest to us more on this later Catalogue more than just a tour d�horizon understand how other professionals approach these tasks self-education if we hope to be part of it Thematic we�ve tried to simplify should tie into interests of our group Project one of these surveys � ESS is of special interest to us more on this later

6. Overview... - screen existing European (and global), representative surveys on their content relevant for the topic of equality of opportunity and intergenerational mobility. - Frank and Erik asked to prepare a note which could be considered the benchmark on which further research should build � e.g. by using the existing results, designing �missing� questions, identifying and targeting �missing� populations and running a �new� questionnaire. - go through methods and organisation of S-o-S - For this purpose: go through 6 different surveys- screen existing European (and global), representative surveys on their content relevant for the topic of equality of opportunity and intergenerational mobility. - Frank and Erik asked to prepare a note which could be considered the benchmark on which further research should build � e.g. by using the existing results, designing �missing� questions, identifying and targeting �missing� populations and running a �new� questionnaire. - go through methods and organisation of S-o-S - For this purpose: go through 6 different surveys

7. Introduction to the S-o-S The European Social Survey (ESS) The European Values Study (EVS) The World Values Survey (WVS) The International Social Survey Programme (ISSP) The Gallup World Poll The European Union Statistics on Income and Living Conditions (EU-SILC) Survey quality of ESS is really good (almost superior � e.g. only face-to-face, sampling should be approved, fixed goal of 70% response rate) , but �relatively little� attitudinal info on EOp That �s why also look at other surveys. Survey quality of ESS is really good (almost superior � e.g. only face-to-face, sampling should be approved, fixed goal of 70% response rate) , but �relatively little� attitudinal info on EOp That �s why also look at other surveys.

8. Introduction to the S-o-S (2) General structure What, when and where? Methodology and sampling How? Availability How to get access to data? What is in it for us? methodology and organisation sampling = aim is random sampling, but actual method of achieving this requirement will vary considerably between participating countries, depending on their access to sampling sources and other considerations. alternative: quota sampling methodology and organisation sampling = aim is random sampling, but actual method of achieving this requirement will vary considerably between participating countries, depending on their access to sampling sources and other considerations. alternative: quota sampling

9. The European Social Survey (ESS) What? How European values and behaviour patterns are distributed, differ within/between countries, change Cross-sectional and longitudinal survey When and where? every two years, from 2002 onwards core module + rotating module Methods and sampling National samples (min 1500 interviews) Aged 15+ Availability ESS data website, without restrictions Cumulative data file Round 1-3 Round 4: by end of September 2009 Social variables including media use, social and public trust; political interest and participation; socio-political orientations, governance and efficacy; moral, political and social values; social exclusion, national, ethnic and religious allegiances; well-being, health and security; demographics and socio-economics. Rotating modules: Round 1 Citizenship, Involvement and Democracy ; Immigration Round 2 Family, Work & Well-Being Opinions on Health & Care Seeking Economic Morality in Europe:�Market Society&Citizenship Round 3 Personal & Social Well-being ; The Timing of Life Round 4 Ageism; Welfare attitudes in a changing Europe Round 5 Work, Family and Well-being Trust in Criminal Justice Cumulative data file Round 1-3 since October 2008 Survey quality of ESS is really good (almost superior � e.g. only face-to-face, sampling should be approved, fixed goal of 70% response rate) , but �relatively little� attitudinal info on EOp That �s why also look at other surveys.Social variables including media use, social and public trust; political interest and participation; socio-political orientations, governance and efficacy; moral, political and social values; social exclusion, national, ethnic and religious allegiances; well-being, health and security; demographics and socio-economics. Rotating modules: Round 1 Citizenship, Involvement and Democracy ; Immigration Round 2 Family, Work & Well-Being Opinions on Health & Care Seeking Economic Morality in Europe:�Market Society&Citizenship Round 3 Personal & Social Well-being ; The Timing of Life Round 4 Ageism; Welfare attitudes in a changing Europe Round 5 Work, Family and Well-being Trust in Criminal Justice Cumulative data file Round 1-3 since October 2008 Survey quality of ESS is really good (almost superior � e.g. only face-to-face, sampling should be approved, fixed goal of 70% response rate) , but �relatively little� attitudinal info on EOp That �s why also look at other surveys.

10. The European Values Study (EVS) What? Investigation of attitudes towards basic human values Cross-national and longitudinal survey When and where? Round 1 1981 14 European countries Round 2 1990 31 European countries + US, Canada Round 3 1999 almost all Europe + � Round 4 2008 46 European countries + � Methods and sampling National samples (target 1000 interviews) Aged 18+ Availability ZACAT, without restrictions Data 2008 : available late 2009/early 2010 Basic human values: family, work, religion, politics and society When: initiated by the European Value Systems Study Group (EVSSG) in the late 1970s. first round (1981), in 14 European countries (Belgium, Denmark, France, Germany, Great Britain, Iceland, Ireland, Italy, Malta, The Netherlands, Northern Ireland, Norway, Spain and Sweden). aroused interest in N and S America, the Middle and Far East, Australia, and South Africa affiliated groups set up to administer the same questionnaire. exchange of data for intercontinental and intercultural comparisons (early start of World Values Studies). second wave launched in 1990 in all European countries (1981 countries + Austria, Belarus, Bulgaria, Czech Republic, Estonia, Finland, Germany East, Hungary, Latvia, Lithuania, Poland, Portugal, Romania, Russia, Slovak Republic, Slovenia, Switzerland), as well as the US and Canada. ten years later (1999), the third EVS wave was conducted in almost all European countries (1990 countries with the exception of Germany, Norway and Switzerland + Croatia, Greece, Luxembourg, Turkey, Ukraine). Alllowed investigation of causes and consequences of dynamics of value change. In 2008, the fourth wave. The fieldwork covered no less than 46 European countries (1999 countries + Albania, Armenia, Azerbaijan, Bosnia-Herzegovina, Cyprus, Germany, Kosovo, Moldova, Montenegro, Norway, Serbia and Switzerland). (FYI: ZACAT is a social science data portal allowing you to search for, browse, analyse and download social science survey data. The studies in this catalogue present a small selection of the studies available at the GESIS Data Archive - however the ZACAT holdings are extended on a regular base. Beside documentation on study level (e.g. abstract, related publications), all studies come along with documentation of full question and answer texts on variable level. Moreover, for some of the collections documentation on variable level is offered in two or more languages. Among others: EVS, ISSP, Eurobarometer) Availability: first three waves: format and contents of the surveys have changed. Variable names are different between surveys, questions have been removed or adjusted and new topics have been added. Online document where you can view variables that are included in the 1981-, 1990- and 1999-datafiles. The document gives an overview of the existing variables and it also shows which of the variables in the three surveys are comparable in time.Basic human values: family, work, religion, politics and society When: initiated by the European Value Systems Study Group (EVSSG) in the late 1970s. first round (1981), in 14 European countries (Belgium, Denmark, France, Germany, Great Britain, Iceland, Ireland, Italy, Malta, The Netherlands, Northern Ireland, Norway, Spain and Sweden). aroused interest in N and S America, the Middle and Far East, Australia, and South Africa affiliated groups set up to administer the same questionnaire. exchange of data for intercontinental and intercultural comparisons (early start of World Values Studies). second wave launched in 1990 in all European countries (1981 countries + Austria, Belarus, Bulgaria, Czech Republic, Estonia, Finland, Germany East, Hungary, Latvia, Lithuania, Poland, Portugal, Romania, Russia, Slovak Republic, Slovenia, Switzerland), as well as the US and Canada. ten years later (1999), the third EVS wave was conducted in almost all European countries (1990 countries with the exception of Germany, Norway and Switzerland + Croatia, Greece, Luxembourg, Turkey, Ukraine). Alllowed investigation of causes and consequences of dynamics of value change. In 2008, the fourth wave. The fieldwork covered no less than 46 European countries (1999 countries + Albania, Armenia, Azerbaijan, Bosnia-Herzegovina, Cyprus, Germany, Kosovo, Moldova, Montenegro, Norway, Serbia and Switzerland). (FYI: ZACAT is a social science data portal allowing you to search for, browse, analyse and download social science survey data. The studies in this catalogue present a small selection of the studies available at the GESIS Data Archive - however the ZACAT holdings are extended on a regular base. Beside documentation on study level (e.g. abstract, related publications), all studies come along with documentation of full question and answer texts on variable level. Moreover, for some of the collections documentation on variable level is offered in two or more languages. Among others: EVS, ISSP, Eurobarometer) Availability: first three waves: format and contents of the surveys have changed. Variable names are different between surveys, questions have been removed or adjusted and new topics have been added. Online document where you can view variables that are included in the 1981-, 1990- and 1999-datafiles. The document gives an overview of the existing variables and it also shows which of the variables in the three surveys are comparable in time.

11. The World Values Survey (WVS) What? Worldwide investigation of socio-cultural and political change Cross-national and longitudinal Together with EVS: 88% of world population When and where? 5 waves from 1981 to 2008 up to 80 countries Methods and sampling Face-to-face interviews Sample: 1000 to 3500 interviewees, aged 18-85 Availability WVS website, without restrictions Also four wave aggregate of WVS/EVS When and where? The World Values Survey first emerged out of the European Values Study (EVS) in 1981, when the methods of a successful European study were extended outside Europe. The 1981 study nevertheless covered only 20 countries worldwide. The survey was repeated after an interval of about 10 years in the second of what came to be termed "waves". One of the aims of the project came to be the longitudinal (as well as cross-cultural) measurement of variation of values. Further waves followed the second wave at intervals of approximately 5 years. Due to the European origin of the project, the early waves of the WVS were eurocentric in emphasis, with especially weak representation in Africa and South-East Asia. To expand, the WVS adopted a strongly decentralised structure. Suitable academic representatives from new countries were free to join in a quasi-democratic network. Joining meant that new representatives had to conduct the predefined survey in their own country with at least 1000 interviewees. They could then exchange their data with the WVS in return for the data from the rest of the project. Funding was primarily local, with representatives in each country funding their own part of the project. In this way the WVS grew out of its eurocentric origins to embrace 42 countries in the 2nd wave, 52 in the 3rd wave, 67 in the 4th wave and 54 in the 5th wave. When and where? The World Values Survey first emerged out of the European Values Study (EVS) in 1981, when the methods of a successful European study were extended outside Europe. The 1981 study nevertheless covered only 20 countries worldwide. The survey was repeated after an interval of about 10 years in the second of what came to be termed "waves". One of the aims of the project came to be the longitudinal (as well as cross-cultural) measurement of variation of values. Further waves followed the second wave at intervals of approximately 5 years. Due to the European origin of the project, the early waves of the WVS were eurocentric in emphasis, with especially weak representation in Africa and South-East Asia. To expand, the WVS adopted a strongly decentralised structure. Suitable academic representatives from new countries were free to join in a quasi-democratic network. Joining meant that new representatives had to conduct the predefined survey in their own country with at least 1000 interviewees. They could then exchange their data with the WVS in return for the data from the rest of the project. Funding was primarily local, with representatives in each country funding their own part of the project. In this way the WVS grew out of its eurocentric origins to embrace 42 countries in the 2nd wave, 52 in the 3rd wave, 67 in the 4th wave and 54 in the 5th wave.

12. The International Social Survey Programme (ISSP) What, when and where? Attitudes towards social issues Supplement to regular national surveys Topical modules on attitudes towards social issues Social inequality 1987, �92, �99, 2009 Role of government 1985, �90, �96, 2006 Family and changing gender roles 1988, �94, 2002 Participating countries vary for each module (recently about 40 countries) Methods and sampling Minimum 1000 achieved sample Mixture of face-to-face and self-completion method Often administered as part of another survey Availability ZACAT, without restrictions What? questions about general attitudes toward various social issues such as the legal system, sex and the economy. Special topics have included the role of government, social inequality, social support, family and gender issues, work orientation, the impact of religious background, behaviour, the environment, beliefs on social and political preferences, and national identity. What? questions about general attitudes toward various social issues such as the legal system, sex and the economy. Special topics have included the role of government, social inequality, social support, family and gender issues, work orientation, the impact of religious background, behaviour, the environment, beliefs on social and political preferences, and national identity.

13. Gallup World Poll What? Reactions to events that have changed the world Cross-sectional and longitudinal survey Organised into performance indices e.g. well-being index When and where? In more than 150 countries, representing 95% of world�s population Methods and sampling face-to-face or by telephone aged 15+ min 1000 interviews per country Availability Gallup WorldView Full Respondent-Level Data info on pricing by mail

14. EU-SILC What? Facts on income, housing, labour, health, demography, education = multidimensional approach of social exclusion cross-sectional and longitudinal replaces ECHP (1994-2001) When and where? from 2003 onwards member states of European Union organised into primary and secondary target variables Methods and sample cross-section: minimal 121 000 households, 250 000 individuals longitudinal: minimal 187 000 individuals aged 16+ Availability micro data for researchers via CD-ROM The European Union Statistics on Income and Living Conditions (EU-SILC) Over an eight year period (from 1994 to 2001), the ECHP (European Community Household Panel), ran in 14 of the then 15 Member States (with the exception of Sweden), and served as the source for many of the commonly agreed social inclusion indicators for this period.From the beginning, the ECHP project suffered from some quality problems, mainly, incomplete geographical coverage, reliability and timeliness. It is with the aims of solving the ECHP technical problems, of conforming to the internationally agreed definition of income and of extending the data collection to the enlarged EU (and beyond) that the decision was taken to stop the ECHP and launch EU-SILC.The European Union Statistics on Income and Living Conditions (EU-SILC) Over an eight year period (from 1994 to 2001), the ECHP (European Community Household Panel), ran in 14 of the then 15 Member States (with the exception of Sweden), and served as the source for many of the commonly agreed social inclusion indicators for this period.

15. Overview...

16. Content: Facts Facts about interviewee Facts about household Facts about father/mother Facts about childhood In grey: variables in pretestIn grey: variables in pretest

17. Equal opportunity in an intergenerational context Equal opportunity in terms of resources Freedom of choice, control, responsibility, effort, luck & equality Role of government Inequality versus mobility �Acceptable reasons� for income differences Content: Values In grey = similar question in pretest Additional remark of Erik: Emphasize that values are attitudes + concepts + perceptionsIn grey = similar question in pretest Additional remark of Erik: Emphasize that values are attitudes + concepts + perceptions

18. Gaps Resources and equality of opportunity both facts and values e.g. �how treated by schooling system and health care?� Information on parents (and grandparents?) e.g. income Childhood environment of the respondents e.g. health, �cultural� environment, neighbourhood, details about educational experiences Meaning of mobility and EOp e.g. Ex ante versus Ex post Representative information on responsibility vs compensation effort and talent: included ; education and health: not included Responsibility of children or responsibility of parents? how do they shift over time?

19. Overview...

20. The Marseille Pretest Relation to existing surveys Some �new� questions� Mobility �Mobile society is a society where �� �I want to live in a more mobile society� �I want my children to live in a more mobile society� High revenues Acceptable that sport stars, singers, advocates have high revenues due to talent and effort? Socially unacceptable that children of rich parents have � What is important for EOp of children? General case, similar to own family, rich family, poor family We need to consider how to exploit the information on �gaps� Erik, Frank + others suggested the way they could be formulated into specific questions On this slide only the topics that were NOT in the existing surveys, but that were in the Marseille pretest Alain will talk about the results But we may want to talk about how these ideas could be developed for he future We need to consider how to exploit the information on �gaps� Erik, Frank + others suggested the way they could be formulated into specific questions On this slide only the topics that were NOT in the existing surveys, but that were in the Marseille pretest Alain will talk about the results But we may want to talk about how these ideas could be developed for he future

21. ESS for us? Applying for rotating modules in ESS Call for proposals placed in Official Journal of the European Union (OJEU) Call for Round 6 later in 2009 or during 2010 likely to be both for both �new� and �repeated� modules documents from Round 5 call are available on ESS website, but will change ! lots of experience working with student samples do we want to pursue a representative-sample study? if so, ESS may be the most promising important to understand how it works lots of experience working with student samples do we want to pursue a representative-sample study? if so, ESS may be the most promising important to understand how it works

22. What Next? Suggestions for further work Allocation of specific topics to different group members A very careful study of the Survey-of-Surveys Theoretical question formulated clearly combination older question/new one Proposal for ESS other �survey providers�? Perhaps connect (1) Allocation of specific topics (2) Study of Survey of surveys Homework approach has worked well for this meeting � and the future? Other survey providers include Gallup who are keen to have interaction with academics Perhaps connect (1) Allocation of specific topics (2) Study of Survey of surveys Homework approach has worked well for this meeting � and the future? Other survey providers include Gallup who are keen to have interaction with academics