Quality Reporting and Communication: Data vs Analytical Interpretation?

1. Session 11: Quality Reporting and Communication Pure data vs. analytical interpretation? Considerations from the perspective of a State Statistical Institute in Germany

2. Starting point • 1. Citation: Johann Hahlen (2010)* • OfficialStatistics increasingly deals with data analysis. Nevertheless, OS is well-advised to be reserved with analysis done for dissemination. Already the selection of • the data for analysis, • the variables, • the reported years for change rates, • the graphs for illustration are decisions not free from value judgements.(Translation U.R. not authorized) „Die amtliche Statistik beschäftigt – sich zunehmend ... auch mit der Analyse von Daten. Sie ist jedoch gut beraten, sich bei der Ergebnisanalyse zurückzuhalten. Denn schon das Zusammenstellen der erhobenen Daten für eine Ergebnispräsentation, die Auswahl der berichteten Merkmale, die Wahl des Zeitraums für eine Veränderungsrate, die graphische Darstellung u. a. sind wertende Entscheidungen.“ (ebenda, S. 376, Hervorhebungen U.R.) * Former DG of NSI Germany, Das Europäische Statistische System im Stresstest Prof. Dr. Ulrike Rockmann

3. Starting point • 2. Citation: Johann Hahlen (2010)* • Official statistics needs a certain amount of analytical competence to provide data services for science and politics. Analysis is always a biased interpretation of selected results in a framework of economic and sociological theories. From there statistical Governance requires that the official statistics limits itself as far as possible to a very objective and neutral data retrieval, so that OS does not endangered the trust in exactness and reliability. (Translation U.R. not authorized) „Sicher braucht die amtliche Statistik eine gewisse Analysekompetenz, damit sie der Wissenschaft und Politik angemessene Datendienstleistungen erbringen kann. Analysen sind indessen immer wertende Interpretation ausgewählter Ergebnisse der beauftragten Statistik vor dem Hintergrund ökonomischer oder soziologischer Theorien. Von daher verlangt statistische Governance, dass sich die amtliche Statistik soweit als möglich auf eine möglichst objektive und neutrale Datenbeschaffung beschränkt, damit sie nicht selbst das Vertrauen in ihre Genauigkeit und Zuverlässigkeit gefährdet.“ (S. 376) * Former DG of NSI Germany, Das Europäische Statistische System im Stresstest Prof. Dr. Ulrike Rockmann

4. Analytical approach: 1. Step in Germany - Legislation 1 • Decision which “data” will be collected • fixed in legislation • is determined by values and goals of the decision makers (… in our case: EU, national government, society, stakeholders) • Decision is not free from value judgements • … but these are not the ones of Official Statistics • Situation in Official Statistics and the Scientific World differs • the decision is not an individual decision of a researcher-team financed by whoever for whatever reason • it is - in democratic societies - a social consensual procedure Optimum outcome: Desired “data” are based on a theoretical framework valid and excepted in the society • Even without further analysis Official Statistics are not free from value judgements • Results (“pure single data”) are reliable in the construct chosen and met 2 1. Step: Legislation 3 4 Prof. Dr. Ulrike Rockmann

5. Analytical approach: 2. Step in Germany – Operationalization 1 Variables are named in law, sometimes with …all, some, non characteristic attributes • if all characteristic attributes are fixed  in general, all possibilities of analysis are fixed and transparent Is it problem that only some analysis is done thru official statistics? • missing characteristic attributes • easy ones – age (day/month/year of birth), etc. • sophisticated ones – immigrant backgroundExample: Children and Youth-Statistics • only the term immigrant background named in the law • NSI (+ Ministries responsible for the topic) decide(s) about characteristic values 2 2. Step: Operationalization 3 4 Prof. Dr. Ulrike Rockmann

6. Analytical approach: 2. Step in Germany – Operationalization 1 • missing characteristic values • apparently easy ones: e.g. highest educational examination • take assignment of German examinations to ISCED • But the assignment table influenced by stakeholders • Nurse (10 years school, 3 years nurse school)  ISCED 5B (ISCED 1997) • Carpenter master craftsman  ISCED 6 (ISCED 2011) If it runs very badly: 2. Step is • biased, not free from value judgements, intransparent step • NSI not independent concerning the methodological decision • Influence by democratically not mandated institutions/persons • Transparency: maybe thru questionnaires & thru publication • Relevance for dissemination!? • What is the responsibility of the NSI in such cases? 2 2. Step: Operationalization 3 4 Prof. Dr. Ulrike Rockmann

7. Life outside …. Analysis using Official Statistic data is done by others - Why should we leave the field to them? • Ineradicable legends • Shouldn’t we actively work against them? • The world changes • Who is the innovator concerning indicators? • How are we going to find indicators that help to answer questions? • What is the life-time of an indicator? • Is Official Statistics only in the role to be “user” of indicators developed by others? • Innovation is necessary and needs … • analysis by the NSI but not only, • transparent communication with the statistical/scientific community as a adjustment factor • Increasing amount complex indicators, methods (census!) and complexity of analysis • Increases the risk of being accused of leaving the „neutral“ level because the understanding decreases – didactical problem! in communication! Prof. Dr. Ulrike Rockmann

8. The Equal Pay Day in Germany – an example why analysis by the NSI is necessary Public perception in Germany: Gap is a result of discrimination against women • NSI-Destatisreporting 18.3.2014[1] • Job characteristics make the difference • Pay gap as result of discontinuity, e.g. baby leave • unadjusted gap: 22% (2010, 2012) • adjusted gap: 7% (2010) • Methodological aspects • “Trend-setter” media reported: • More time-limited formats: only unadjusted • Quote economic institutions with lower gaps Prof. Dr. Ulrike Rockmann

9. Summing up • The situation • My point of view: there are no “pure data” = free from value judgements • In the media: all data are lumped together and are called “Statistics” • Official statistics has to legitimate itself often • The image of Official Statistics in the public is indifferent (in Germany) • “Self-motivated” analysis is with respect to the special responsibility – as a part of democracy - necessary • otherwise Official Statistics can’t be innovative • Is only reactive and not active • Publish in reviewed papers • Establish the statistical community as an adjustment factor (find a better link to the existing) • “Keep away” from “depended” analysis • Public perception • does not link official statistics with scientific work  Necessity of intensifying activities to educate the recipients Prof. Dr. Ulrike Rockmann