Learning Objectives

1. Workshop on Improving Gender Statistics in RwandaSession 5Statistics’ Quality and Comparability: Metadata and International ComparisonsSerena Lake Kivu Hotel, Rubavu DistrictMarch 25-27, 2014

2. Learning Objectives At the completion of this module, participants should be familiar with: • What are ‘metadata’ and why are they important? • Examples of metadata that can be used for gender indicators • Suggestions for producing the GSF metadata • Why and How to compare gender statistics internationally • Current Problems with International Data Comparability • International mechanisms for keeping informed about developments on gender statistics Sources: Gardner, Jessica. “Importance of metadata.” Workshop on Writing Metadata for Development Indicators Lusaka, Zambia, 30 July – 1 August 2012, UNECA and African Union Oakley, Graeme, Australian Bureau of Statistics. www.unescap.org/stat/apex/2/APEX2_S.4_conference_Statistical%20Metadata%20Standards.pdf; OECD, Management of Statistical Metadata at the OECD, V/ 2.0, 6/9/2006.http://www.oecd.org/std/33869551.pdf UN Statistics Division, Department of Economic and Social Affairs. Millennium Development Goals Indicators. Series Metadata, http://mdgs.un.org/unsd/mdg/metadata.aspx World Bank.

3. Part 1. Metadata for gender Statistics

4. What are Metadata? “Metadata: the range of information, generally textual, that fosters understanding of the context in which statistical data have been collected, processed and analyzed with the objective of creating statistical information…” African Charter for Statistics (2009) “Metadataprovide information on data and about processes of producing and using data. Metadata describe statistical data and - to some extent - processes and tools involved in the production and usage of statistical data.” UNECE, "Guidelines for the Modeling of Statistical Data and Metadata“ (1995). • Provide information that defines or describes the data or statistics • Describe the data collection, production, processing, computation and analysis process as well as the content and source of the data • Also discuss the limitations and quality of the data • Created and used throughout the data production process • Respond to and inform national standards and systems

5. Metadata include information not only about the “ingredients” (components) but also about how the data were produced, i.e., the process

6. Why Metadata are important for reporting on Gender and Development Indicators • Inform users about the source, definition, collection process and limitations of the data • Provide users with knowledge and understanding of statistics availability and use • Clarify indicators from multiple data sources with different definitions, data collection process, dates, etc. • Explain discrepancies in indicators’ estimates--e.g., for the MDGs • Guide new data collection and statistics’ and indicators’ production

7. Example 1. Metadata for Rwanda GSF Indicators: What to look for • There is no international consensus about what appropriate metadata should contain -- different producers include different information items • Example 1: OECD - List of common metadata items: Source: OECD, Management of Statistical Metadata at the OECD, V/ 2.0, 6/9/2006. http://www.oecd.org/std/33869551.pdf

8. Example 2: World Bank’s World Development Indicators Metadata • The metadata for the World Bank’s World Development Indicators includes the following: • Code • Development relevance • Statistical concept and methodology • Indicator Name • Long definition • Source • Topic • Periodicity • Aggregation method • Limitations and exceptions • General comments • Handout 5.1 has an example of the WDI metadata for Under-5 mortality rate • This same metadata is being developed for the Bank’s Gender Statistics database. Source: World Bank, World DataBank. World Development Indicators. http://databank.worldbank.org/data/views/variableselection/selectvariables.aspx?source=world-development-indicators

9. Example 3. Millennium Development Goals (MDGs) Indicators Metadata • The metadata for the MDGs includes the following items: • Contact point in the international agency that produces the data • Definition • Method of computation • Comments and limitations • Sources of discrepancies between global and national figures • Process of obtaining data • Treatment of missing values • Data availability • Regional and Global estimates • Expected time of release Source: UN Statistics Division, Department of Economic and Social Affairs. Millennium Development Goals Indicators. Series Metadata, http://mdgs.un.org/unsd/mdg/metadata.aspx

10. Example 5: Metadata for the UN Minimum Set of Gender Indicators (MSGI) • UNSD has compiled the metadata for the set based on information from data collection agencies: WHO, UNESCO, ILO, etc. • The data and metadata are provided by international agencies based on national data or information reported to them • Some data are old or are estimates or projections produced by the international agencies • Rwanda has more recent data that is not included in the MSGI • Metadata are available for 46 indicators; all but 2 are Tier I indicators (they meet the 3 criteria discussed yesterday) • The information is the result of consultation and agreement among the data collection agencies. • The metadata highlight the problems and limitations of the data for international comparability; • Some problems or limitations highlighted for the international level may not be relevant at the national data because there may be more detailed information about the data.

11. Metadata for the UN Minimum Set of Gender Indicators (2) • The metadata contain information on 10 areas—similar to MDGs metadata: • Indicator Name: • Contact point in international agency • International agreed definition • Method of computation • Importance of the indicator in addressing gender issues and its limitation • Sources of discrepancies between global and national figures • Process of obtaining data • Treatment of missing values • Data availability and assessment of countries’ capacity • Expected time of release • For Rwanda, the UN MSGI has data for 34 of the 52 indicators, although there are no metadata for 2 of them – see Handout 5.2 • UN MSGI metadata can serve as a source or example for producing GSF metadata: • They can help improve the collection and quality of the gender indicators and statistics, • But, they may not be suitable for the GSF because they were produced for the international and not the national level – they provide information that is not relevant at the national level

12. Rwanda GSF Indicators: examples Without Metadata – • Gender parity index for secondary gross enrolment (females to males): • 1.00 (Rwanda Education Statistics, EMIS, 2010) • 1.04 (Rwanda Education Statistics, EMIS, 2011) • Literacy rate among population aged 15-24, by sex (Youth Literacy rate): • Female: 85% (DHS 2010) • Male: 83% (DHS 2010) • Life expectancy at birth (years): • Female: 54.8 (Rwanda Population Projection, 2009) • Male: 50.8 (Rwanda Population Projection, 2009) • Maternal mortality ratio, MMR (maternal deaths per 100,000 live births: • 476 (DHS 2010)

13. Metadata for Rwanda GSF Indicators: Gender parity indexfor secondary gross enrolment (females to males) With Metadata • Gender parity index for secondary gross enrolment: • Definition: Index is the ratio of female to male gross enrollment ratios in secondary education. • Gross enrolment ratio, GER: total enrollment in secondary education, regardless of age, expressed as the percentage of the population of official secondary education age (World Bank Metadata). Can be calculated separately for females and males. • Numerator: population enrolled in secondary school • Denominator: total population of official secondary school age • GER can exceed 100% due to the inclusion of over-aged and under-aged students because of early or late school entrance and grade repetition (World Bank WDI Metadata). • Unit: ratio • Computation: • Numerator: GER Females (enrolled females as a % of all females of secondary school age) • Denominator: GER males (enrolled males as a % of all males of secondary school age) • Measurement/Estimation: GER females/GER males. • Source: Administrative data—Rwanda Education Statistics, January 2011 • When and how were the data actually collected? • Lead Agency/producer: Ministry of Education?

14. Metadata for Rwanda GSF Indicators: Literacy rate among population aged 15-24, by sex (Youth Literacy rate): With Metadata • Definition:The percentage of the population aged 15–24 years who can both read and write with understanding a short simple statement on everyday life. (Source: UIS) • Unit: % • Computation: Literacy rates are computed by dividing the number of persons [females or males] aged 15-24 years who are literate by the total [female or male] population in the same age group. The result is then multiplied by 100 to yield the literacy rate in per cent. (Source: UIS) • Importance of the indicator in addressing gender issues and its limitation: The Youth Literacy Rate reflects the outcome of primary education over the previous decade. As a measure of the effectiveness of the education system, it is often seen as a proxy measure of social progress and economic achievement. Reasons for failing to achieve the literacy standard may include non-attendance, low quality of schooling or dropping out before completion of primary education. Differences in literacy levels between young women and men will often reflect recent inequalities in access to formal education and persisting inequalities in adult life and the world of work. (Source: UIS) • Source: DHS 2010 • Lead Agency/producer:Measure DHS?

15. Metadata for Rwanda GSF Indicators: Life expectancy at birth by sex With Metadata – • Life expectancy at birth: • Definition:Estimate of the average number of years a newborn is expected to live based on current age-specific mortality rates. • Life expectancy at birth by sex gives a statistical summary of current differences in male and female mortality across all ages. • In areas with high infant and child mortality, the indicator is strongly influenced by trends and differentials in infant and child mortality (Source: UN DESA, Population Division, 2011). • Unit: Number of years • Year/Date: Calendar year when data were collected -- When and how were the data actually collected? • Source: Rwanda Population Projection 2009 – Based on which data sources: census, survey, administrative records, several sources? • Assumption: Current age specific death rates/mortality patterns will remain constant in the future • Lead Agency/producer: NISR?

16. Metadata for the Rwanda Maternal Mortality Ratio, MMR • Definition: Annual number of female deaths from any cause related to or aggravated by pregnancy or its management (excluding accidental or incidental causes) during pregnancy and childbirth or within 42 days of termination of pregnancy, irrespective of the duration and site of the pregnancy, per 100,000 live births, for a specified year (WHO). • Unit: Ratio • Computation: • Numerator: For Rwanda: Any death that occurred during pregnancy, childbirth, or within two months after the birth or termination of a pregnancy. Includes all deaths occurring during the specified period even if due to causes that are not pregnancy related (DHS 2010). • Denominator: International convention--Number of live births (in 100,000s), based on either a written record or the mother's recall. • MMR for Rwanda-DHS 2010: Expressed per 100,000 live births; calculated as the maternal mortality rate divided by the age-adjusted general fertility rate, which is the average number of live births per 1,000 women of reproductive age (age 15-44) (DHS 2010). • Year/date: Fieldwork was conducted September 2010-March 2011 (DHS 2010). • Measurement/estimation: For Rwanda: Women respondents reported the number of their sisters who died, and the number who died of maternity-related causes. No definitive procedure for establishing completeness or accuracy of retrospective data on sibling survivorship (DHS 2010). • Source: DHS 2010. • Lead Agency: MEASURE DHS (previously Macro International) • Limitations of the indicator: Based on recall of deaths and live births by key informant (mother’s sister or mother). DHS uses a different denominator to overcome the limitations of the recall.

17. Suggested steps for producing metadata for the GSF indicators: Not a prescriptive or exhaustive list: • Decide who to partner with on this work – e.g., • NISR metadata initiative • Possible pilot with one data collection exercise-e.g., survey • Decide which template to use • Adopt features from different examples and customize them for Rwanda’s needs • May involve consultation with data producers and users • Explore experiences from other countries in Africa or developing countries • Decide which indicators to focus on first: main ones, by sector, or other criteria • GSF has hundreds of indicators so it will take a lot of time and work to produce metadata for all • Consider piloting in one sector or one data collection initiative • Go to the data sources for Rwanda to find out information about data collection process, definitions used, limitations and problems, coverage and response rate, collection dates, etc. • May involve reviewing data collection manuals, templates, questionnaires and consulting with staff involved in the process

18. Suggestions for producing metadata for the GSF indicators • Complement with information from international sources when appropriate • e.g., internationally agreed definitions, importance from a gender perspective, limitations, comparability • Consult and negotiate with data producers and compilers about feasibility of using the framework or template in censuses, surveys, administrative data collection – • NISR, line ministries, civil registration personnel, international agencies • This may involve an iterative and continuous process to adjust and improve the template • Agree on and harmonize the metadata with all the data producers • Publish and circulate the metadata to all producers and users • Train data collectors on how to produce and report the metadata • Training may need to be provided to data collectors, compilers and reporters every time a new data collection is started

19. Disseminated data should always be accompanied by metadata Important! • Disseminated data should be accompanied by metadata to help users understand the data. • Can be included after the data (each section), as an annex, or as links to the indicators in electronic format • Metadata should include, but is not limited to, information on: • Concepts, definitions and classifications used • Basic features of the data sources • Data collection methodology: censuses, surveys, administrative records • Guidelines on use of the data • Data quality (e.g. sampling and non-sampling error, non-response rates, data comparability). • In some circumstances, it can be useful to release particular types of metadata in a dedicated publication. • For example, in Vietnam, the GSO published a Gender Statistical Handbook in 2011.

20. Part 2. Comparing Gender Statistics Internationally

21. Benefits of International Comparabilityof Gender Statistics • Similarities and differences in gender issues between individual countries and between regions can be studied and relative progress on gender-related goals can be assessed • by undertaking data comparisons across countries or regions. • The overall quality of a country’s statistics can be enhanced • because producing comparable statistics involves adoption of international standards and best practice in methodology. • Gender issues and developments can be analysed in an international context • by combining statistics across countries to produce regional and global aggregates.

22. Bringing gender statistics from different countries together: • The World Bank’ Gender Statistics electronic database • http://datatopics.worldbank.org/gender/ • Allows users to compare statistics on gender for regions and countries in 6 areas: economic structures and access to resources; education; health and related services; public life and decision-making; and human rights of women and girl children. • The data come from the World Development Indicators and additional sources. • Users can create their own tables and download the data into Excel • The UNSD publication The World’s Women 2010, Trends and Statistics • highlights the differences between the status of women and men in various areas of contemporary life. • covers 196 countries across the world. • presents and analyses data at global, regional, and individual country levels. • http://unstats.un.org/unsd/demographic/products/Worldswomen/Executive%20summary.htm • The following charts from that publication illustrate how country data can be brought together to inform gender issues in a wider context.

23. Global sex distribution Source: UNSD The World’s Women 2010: Trends and Statistics

24. Regional sex distribution Source: UNSD The World’s Women 2010: Trends and Statistics

25. Time spent in domestic work per day, 1999-2008 Source: UNSD The World’s Women 2010: Trends and Statistics

26. Employed persons in vulnerable employment by region and sex, 2004-2007 Source: UNSD The World’s Women 2010: Trends and Statistics

27. Current Problems with International Data Comparability • Gaps in the availability of gender statistics in many countries • For example, in 12 of the 22 topic areas specified in the 2012 UNSD Global Review of Gender Statistics Programmes, more than a quarter of 126 countries reviewed were not producing any gender statistics. • Lack of comparability in many of the gender statistics that are available for individual countries • International standards for producing comparable gender statistics are not available or incomplete for some topics • Existing international standards are not always fully implemented by countries • For example, a country may consider that a particular standard classification is not useful, impractical or inappropriate in its circumstances and therefore not adopt the standard.

28. International mechanisms for keeping informed about developments on gender statistics • United Nations Statistical Commission (UNSC)annual meetings • Reports on gender statistics produced and available for all meetings in the last few years. • UNSD Global Forum on Gender Statistics biennial meetings • A range of documents, presentations and reports are publicly available. • The last Global Forum was held in Jordan in March 2012 and focused on women’s empowerment; • Next Global Forum is planned for 2014 and expected to focus on gender analysis and use of gender data and indicators. • UNSD regional meetings and training workshops • UNECA meetings

29. Exercise 5.1: Group Activity • Provide several examples that illustrate the value of producing gender statistics for Rwanda that are comparable with those of other countries. • What are the main obstacles to producing internationally comparable data on gender statistics? • What actions would need to be taken, and by whom, to ensure gender statistics in Rwanda are internationally comparable?