The Future of Social Statistics

1. The Future of Social Statistics Martine Durand OECD Chief Statistician Seminar on Shifting Paradigms New York, 18 February, 2011

2. Overview • Where have we come from? • How has the OECD responded? • The current environment : where are we now? • Which set of indicators? • Challenges ahead

3. Where have we come from? “The focus of social indicators is to measure levels of living, and social and economic factors considered to influence levels of living”, UN Handbook on Social Indicators, 1989 “A sound system of social statistics is vital to the effective development of social policy, to informed decision-making on policy issues, and to the evaluation of the impact of social and economic policy”,UNSC Expert Group, 1996 “There is a need for more internationally agreed statistical standards and guidelines, and for improved frameworks in which to summarize policy outcomes”, UNSC Expert Group, 1996

4. Where have we come from? Social statistics and indicators are about... • both a discrete set of domains (demography, employment, health, education, social and family policies, welfare systems, physical security, etc...) • and a specific way of looking at these and other (economic) domains, focusing on differencesin conditions among population groups and on distributions rather than on aggregates only • household income is an economic statistic • income gap between men and women is (also) a social statistic • the distribution of household income is (also) a social statistic

5. Where have we come from? ... Social statistics also combine different perspectives... Outcomes (skills, poverty, health status) Outputs (beneficiaries, take-up rates, benefit levels, graduation rates) Inputs and processes (spending, programme features) Context (demography, socio-economic characteristics, labour market)

6. Where have we come from? ... Importantly, social statistics are about PEOPLE • basic demographic and contextual statistics are the first things any government collects... • Oldest surviving Census data comes from China (Han Dynasty, 2AD ) • The Doomsday Book (commissioned by William the Conqueror in 1086)

7. Where have we come from? • Andoutcome measurement has a long history, predating that of official economic statistics • “Life and Labour of the People” (1889); statistical analysis of poverty in London, Charles Booth • “Recent Social Trends in the US” (1933); two-volume report released by the Hoover Committee on Social Trends, chaired by Wesley C. Mitchell (NBER)

8. Where have we come from? Statistical map of Whitechapel, London, 1886 Red is upper socio-economic areas Black is “semi-criminal” areas Source: Charles Booth, 1889

9. Where have we come from? • Despite long-standing interest in social statistics ... • ...Still widespread perception that social statistics are ‘soft’ as compared to ‘hard’ economic statistics because they lack an additive and integrated accounting structure such as the SNA

10. Where have we come from? • Following the development of the SNA, there have been several attempts to develop frameworks for social statistics: • System of Social and Demographic Statistics, UNSD • Social Indicators Movement

11. Where have we come from? • System of Social and Demographic Statistics (1975) prepared by Richard Stone for UNSD • Ambition: to develop a ‘system’ for social statistics matching the SNA • How: to create an accounting framework for social statistics focusing on flows between different states • Objective: measures of inputs, outputs, and outcomes in a single framework

12. Where have we come from? • The attempt failed due to: • Differences among countries in state of development of social statistics and national priorities • No agreement on common concepts, definitions and measurement instruments • Lack of a single common metric for integration and aggregation of fundamentally different domains • Cost of collecting some of the data • Lack of clear policy relevance for a social accounting framework

13. Where have we come from? • The Social Indicators Movement in the 1970s • strong push towards measuring social outcomes in a more systematic way (e.g. social contacts, health status, time use) • did not attempt to produce an over-arching accounting framework for all social activities

14. Where have we come from? • In the 1970s,the OECD alsohad a significant work programme around social indicators: • Measuring Social Well-Being: A Progress Report on the Development of Social Indicators, OECD, 1976. • Social Accounting Matrices • introduced extra detail for specific socio-economic issues into the national accounts framework

15. Where have we come from? • These attempts also failed to produce a stable set of social statistics • despite best intentions, many indicators were still input-based or of too limited scope with no overall framework for guiding social policy and tracking the impact of social programmes • the key policy challenges of the 1980s were largely seen to be economic rather than social in nature and social indicators were therefore viewed as a ‘luxury’ rather than a necessity

16. Where have we come from? • Important lessons to be learned : • crucial to articulate what is being measured and why to be useful for policy decision-making • limited applicability of accounting and integrated frameworks to social statistics

17. Where have we come from ? Integrated framework for economic statistics Framework for social statistics Some policy domains are interconnected (e.g. employment , gender gaps, poverty), but not all. Difficult to monitor “overall” social policy No single metric (and controversies on valuation for non-market activities) Minimum set of subject-matter statistics and indicators • Strong linkages between economic statistics and economic policy issues (e.g. labour, prices, national accounts, government finance and balance of payments) allowing economic policy monitoring • Single (additive) metric • Integrated accounts

18. Where have we come from? • Important lessons to be learned : • crucial to articulate what is being measured and why to be useful for policy decision-making • limited applicability of accounting and integrated frameworks to social statistics • frameworks for social statistics need to be sufficiently flexible to be adapted to countries with different social concerns better to develop an analytical rather than an accounting framework

19. How has the OECD responded? • Since the late 1980s, the OECD’s response has therefore been pragmatic and policy focused: • Developing indicators linked to clear policy objectives in specific domains

20. How has the OECD responded? • In several areas, the OECD has developed a reliable reporting system based on comparable concepts and definitions, e.g.: • Social expenditure database (SOCX) • Health Data, System of Health Accounts • PISA, Education Indicators • Family Database • Working in close collaboration with national experts (INES, Health data correspondents) • Covering contextual, input, output and outcome indicators

21. How has the OECD responded? • Social expenditures consistent reporting of public and private social expenditures in key domains Public and private social expenditure as percentage of GDP in 2007

22. How has the OECD responded? • Health care quality indicators: comparison of health systems performance across a number of key dimensions

23. How has the OECD responded? PISA: measures of standardised performance in reading science and math among students aged 15

24. How has the OECD responded? • Family Database: an on-line database of family outcomes and policies

25. The current environment • Renewed interest in measuring and monitoring outcomes in a more encompassing/less discrete way: • International initiatives (MDGs, HDI, OECD Measuring Progress, Stiglitz report and follow-up; UNECE/OECD/Eurostat Task Force on Sustainable Development) • National initiatives (e.g. State of the USA, Measuring Australia’s Progress, UK National Well-Being, etc) • Go beyond social field (economic; environmental) • In contrast to the Social Indicators Movement, most current initiatives have a coherent analytical framework indicating what is being measured and why

26. The current environment • Interest in measuring social outcomes is driven by: • citizens’ and governments’ strong demand for more information on what really matters to people beyond economic aggregates • evidence-based policy and the use of measurable policy targetsfor evaluations and impact assessments • increased ability to measure key outcomes that were regarded as inherently un-measurable only a few years ago • cross-country comparisons of performance as populations are tied more closely together through globalisation and governments wish to understand better the roles of institutions and drivers

27. Where are we now? • Is the progress/well-being measurement agenda a new paradigm for social statistics? • Can it represent the new unifying framework for social statistics?

28. Where are we now? Measuring progress: an encompassing framework focusing on people’s well-being and outcomes Materialwell-being Regrettables GDP People’s overall well-being Capital stocksPhysical, natural, human, social

29. Where are we now? • Similar to, butbroader in scope than, the Social Indicators Movement : • Not only what happens in society from the perspective of governments, but also what happens in governments from the perspective of citizens • Not only conditions of people but also those of the environment where people live (exposure to environmental hazards, pollution, housing) as they affect their well-being • Not only people’s objective conditions but also subjective feelings about them

30. Where are we now? • It DOES NOT provide a comprehensive unifying accounting framework for all social statistics as Richard Stone attempted in the 1970s… • …but it DOES provide a clear conceptual framework for outcomes indicators

31. Which set of indicators? What outcome indicators are we talking about? • measures that are relevant for a broad range of societies but can be ‘declined’ in different ways to reflect national concerns • measures that are relevant for designing evidence-based policies • a tool for social monitoring and reporting (needed for periodic medical check-up to assess the patient’s condition)

32. Which set of indicators ? • Are these measures in competition with other initiatives (MDGs, EU2020, targets in national planning documents, e.g. India)? • The answer is NO : • Progress measures : the entire car dashboard (e.g. indicators of speed, gasoline left, oil pressure, road conditions), i.e. continuous reporting based on a comprehensive set of measures • MDGs : focus more on “red light” indicators on the dashboard signalling ‘critical conditions’ (e.g. lack of gasoline) and need for immediate action

33. Which set of indicators? • What needs to be done? Recommendations from the Stiglitz-Sen-Fitoussi Commission : • Extend the measurements of economic resources, putting more emphasis on the resources of households and their distribution • Develop measurements of subjective well-being and quality of life, as well as of inequalities in their distribution • Produce information that allows better assessments of the sustainability of growth, [and develop indicators of environmental impact of economic activities]

34. What is the OECD doing? • During OECD 50th Anniversary (“better policies for better lives”), publication of “How’s Life?” report • Accompanied by a number of methodological projects done under the auspices of CSTAT, in collaboration with other international organisations • Aim is to improve existing indicators and build new ones

35. What is the OECD doing? • “How’s Life?” will present existing comparable indicators for OECD and selected emerging countries (the “low hanging fruits”) • “How is Life?” draws on the SSF recommendations • primary focus is on measuring people’s well-being • Material conditions • Quality of life • secondary focus is on measuring sustainability • Key capital stocks • Work in progress: will evolve over time

36. What is the OECD doing? Well-Being Dimensions Material well-being Quality of life Health status Education and skills Work and life balance Civic engagement and governance Social connections Environmental quality Personal security Subjective well-being • Income and wealth • Jobs and earnings • Housing

37. What is the OECD doing? • Choice of indicators under each dimension based on common set of criteria: • Validity/Relevance(of the available measures w.r.t. the outcomes of interest?; are they amenable to policy change?) • Cross-country comparability of information • Depth(information on distribution of outcomes) • Well-established sources (is the information from official/reliable sources?) • Recurrence of data collection • Indicators assessed against each of these criteria: adequate (√), limited (~) and missing (X)

38. What is the OECD doing?

39. Challenges ahead • Work required to build a common/minimum set of headline indicators • Need to fill in statistical gaps, but: • National statistical systems are under pressure to deliver progress/well-being indicators at the same time as NSOs’ budgets are being squeezed and priorities in social fields conflict with other domains

40. Challenges ahead • Developing relevant indicators in new or under-developed domains • In some domains indicators are not available or only provide averages or only cover specific at-risk populations • Need to build new valid indicators based on common definitions and concepts; important to establish priorities and timeframes Way forward: UNECE /Eurostat/OECD TF work on SD; INSEE/Eurostat Sponsorship; UN Regional Commissions; launch new research projects ?

41. Challenges ahead • The case of subjective indicators • For some aspects of well-being both perceptions and reality matter for analysis and policies; need for objective indicators and information on values that people attach to various dimensions • Most NSOs already collect subjective information in some of their surveys (General Social Survey, Time Use Survey), but this information is often not comprehensive and not comparable internationally Way forward: OECD Guidelines for developing SWB indicators in collaboration with Eurostat/EU SILC, EQLS, UK, US

42. Challenges ahead • Official v.s. non-official sources • In some domains, the only type of comparative information that currently exists is through non-official sources Way forward: • Expand the scope of official instruments: Household Surveys; LFS; Time Use Surveys; EU SILC • Use non-official sources if NSOs judge that data are of sufficient quality and cheaper; work with (semi) private providers (e.g. EQLS) ?

43. Challenges ahead • Depth and Consistency : use of micro-data and common standards and definitions • Matching of various survey-based and administrative data to assess outcomes for population subgroups, and to allow analysis of causality and policy impacts • Make micro data available to research community • Differences in underlying concepts and definitions limit integration at country-level and cross-country comparability Way forward: OECD expert groups on joint distribution of income, consumption and wealth; on integrating inequalities in NA (with Eurostat); on micro- data access

44. Challenges ahead • The challenges just outlined require serious methodological work in a number of areas • A number of IOs and countries have started new projects that will address some of them • Those need to be shared and coordinated to ensure maximum coherence and consistency in standards and definitions across instruments used to collect indicators

45. Conclusions • Past international attempts to design a fully integrated framework for social statistics have failed • More pragmatic approach restricted to specific domains has had some success both at country and international level • Today, there is renewed and strong interest for a more encompassing framework focusing on outcomes • Measuring Progress and SSF follow-up represent a great opportunity to move ahead • Not without challenges, but these can be addressed