Indicators, data and country level reporting: lessons for monitoring

KÖZIGAZGATÁSI ÉS IGAZSÁGÜGYI MINISZTÉRIUM TÁRSADALMI FELZÁRKÓZÁSÉRT FELELŐS ÁLLAMTITKÁRSÁG Indicators, data and country level reporting: lessons for monitoring Expert meeting - Budapest, 28 April 2011 István György Tóth

THANKS TO ALL SPEAKERS AND PARTICIPANTS FOR ALL THE VERY USEFUL COMMENTS !!!

Major dilemmas - Shall we be more ambitious or less ambitios?? • Shall we evaulate or shall we not do that?

Key aspects of next steps • The report text • The indicator development process • The scheme/format of presentation • Data infrastructure • (The politics/policy of regular monitoring)

Time horizons • By the end of the current contract deadline (next two weeks) • Medium term: next half a year, one year … ?)

Tasks: the report texts, next two weeks • Implement changes suggested by the participants of the expert seminar (what can be done consistently) • More careful evaluations, use the five steps suggested by Chris and document policy relevance of the new indicators • Re-calibrate evaluation thresholds in Policy Marker Report Cards (rather than simply 30%), together with statistical significance • BETTER REFERENCING and FULL RECOGNITION OF THE PREVIOUS WORKS and also to • Explicit reference to UNCRC • Connect to EU2020 and to the legally adopted (2010 dec) joint assessment framework (theme child poverty) • Reflect, wherever possible, institutional country specificities in some relevant dimensions (for childcare variables, for example) • Connect to ISG terminology • Mention caveats why we do not use socia participation and local environment variables • Datasets documentation (from expert studies)

Tasks: the report texts, next two weeks (contnd) • Complete the analysis • Draw main conclusions, prepare executive summary • Copy-edit, finalise, dissemination plan • Make clear the status of this report: this is a statistical/analytical paper (rather than a political, though with policy focus). This is not an „official” presidency document even!!

Tasks: indicator development, medium term (to be summarized in exec summary) • Complete „still-to-be-dones” outlined in main presentation on the basis of data available • Special attention to further integrate EU2020 indicators • See what can be done re the two unexplored dimensions (local environment and participation) • Analyse 2009 EU-SILC module and implement results (make suggestions for regularly monitored indicators) • Explore options for improvements in dimensions not covered with sufficiently standardised data collections (crime, institutionalised care, most vulnarable in general, etc) • KEEP PROFESSIONAL STANDARDS AND DO NOT USE LESS RELIABLE, NOT ENOUGH TIMELY, NOT POLICY RELEVANT ETC INDICATORS!!

Tasks (for further developments) re scheme/format of country reporting, medium term • Scorecard or not a scorecard??? Pluses and minuses or not? Let we make a policy marker report card to both keep the policy signal aim and try avoid government officials’ „comparison phobia” of naming and shaming!!! This helps develop a context in which governments compare themselves to achievements and not to each other!! • Refine the structure of policy marker report cards: this is a first try of the experiment with this approach • Further validate/reconsider actual lead indicators • Refine/calibrate evaluation methods (what benchmarks, what averages, what thresholds) • Improve on country specific data deficits • Maintain independent professional control of the production of annual policy marker report cards

Data infrastructure (these to be mentioned in the report as summary recommendations) • Collect context information on child and family related social expenditures, within the OMC reporting routines • Integrate suggested new indicators into regular data collection instruments (like education deprivation indicators into EU-SILC) • Investigate improved, regular microdata access to various valuable ongoing data collection excercises (most notably HBSC, ESPAD and others) • Explore on improving incentives to support substitute or alternative datasets in national contexts • Explore on options for monitoring the social situation of the children of migrants and Roma • Further investigate the potential for utilising national administrative datasets • Find out ways to monitor indicators/dimensions where data standardisation is problematic (crime statistics, children in institutionalised care, etc) • Invest in panel surveys (national or EU level) to facilitate exploring causal relationships • Involve researchers in questionnaire development

(The politics/policy of regular monitoring) • Commission position on next steps? (recommendation in 2011) • Balance for less developed countries (emphasised by A.C) • In addition to the foreseen adoption of the new child well being indicators, what other SPC/ISG plans for the comprehensive indicator portfolio (including new breakdowns and health education and risk behavior indicators)? • Relationship of Social OMC and EU2020 monitoring and the role of child wellbeing in these? .. And to the European platform against poverty? • Cont(r)act major data infrastructures, ensure continued supply of good quality data • Stakeholder involvement/ownership in keeping momentum? • Forthcoming presidency activities?

Thanks again

Indicators, data and country level reporting: lessons for monitoring