1 / 14

DATA QUALITY

DATA QUALITY. L-MEP. Liberia - Monitoring & Evaluation Program. How closely do the data used reflect the truth about results?. DATA QUALITY: OBJECTIVES. Understand the 5 Data Quality Standards Apply a Data Quality Assessment Tool. L-MEP. Liberia - Monitoring & Evaluation Program. 2.

hien
Download Presentation

DATA QUALITY

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. DATA QUALITY L-MEP Liberia - Monitoring & Evaluation Program How closely do the data used reflect the truth about results?

  2. DATA QUALITY: OBJECTIVES Understand the 5 Data Quality Standards Apply a Data Quality Assessment Tool L-MEP Liberia - Monitoring & Evaluation Program 2

  3. Issues MANAGEMENT Can you make decisions based on the data? Better quality data leads to better informed management and planning. REPORTING Are the data believable? Audiences want to know how “credible” your data are so they can trust your analysis and conclusions. L-MEP Liberia - Monitoring & Evaluation Program 3

  4. Five standards for quality of data L-MEP Liberia - Monitoring & Evaluation Program VALIDITY RELIABILITY TIMELINESS PRECISION INTEGRITY

  5. Validity Key question: Do data clearly and directly measure what we intend? (7 indicator characteristics?) – L-MEP Liberia - Monitoring & Evaluation Program Issue: Bias • Result: Modern sanitation practices improved • Indicator: Number of residents in targeted villages who report using “clean household” practices • Source: door-to-door survey conducted three times a year • Most of the people in the targeted region work long hours in the fields during the harvest season Issue: Direct • Result: Poverty of vulnerable communities in conflict region reduced • Indicator: Number of people living in poverty • Source: government statistics office • The government doesn’t include internally displaced people (IDPs) in the poverty statistics 5 5

  6. ReliabilityKey question: If you repeated the same measurement or collection process, would you get the same data? L-MEP Liberia - Monitoring & Evaluation Program Issue: Consistency or Repeatability • Result: Employment opportunities for targeted sectors expanded • Indicator: Number of people employed by USAID-assisted enterprises • Source: Structured interviews with USAID-assisted enterprises, as reported by implementing partner AAA, BBB, and CCC • The AO Team found out that the implementing partners were using these definitions: • AAA – employees means receives wages from the enterprise • BBB – employees means receives full-time wages from the enterprise • CCC – employees means works at least 25 hours a week 6 6

  7. Timeliness Key question: Are data available timely enough to inform management decisions? L-MEP Liberia - Monitoring & Evaluation Program Issue: How Frequent • Result: Use of modern contraceptives by targeted population increased • Indicator: Number of married women of reproductive age reporting using modern contraceptives (CPR) • Source: DHS Survey • The DHS survey is conducted approximately every 5 years Issue: How Current • Result: Primary school attrition in targeted region reduced • Indicator: Rate of student attrition for years 1 and 2 at targeted schools • Source: Enrollment analysis report from Ministry of Education • In July 2002 the MOE published full enrollment analysis for school year August 2000 – June 2001 7

  8. Precision Key question: Are the data precise enough to inform management decisions? L-MEP Liberia - Monitoring & Evaluation Program Issue: Enough detail • Result: CSO representation of citizen interests at national level increased • Indicator: Average score of USAID-assisted CSOs on the CSO Advocacy Index • Source: Ratings made by Partner XXX after interviews with each CSO • The SO team reported this data to the Mission Director: • 1999 = 2.422000 = 32001 = 3.000 Issue: Margin of error • Result: Primary school attrition in targeted region reduced • Indicator: Rate of student attrition for years 1 and 2 at targeted schools • Source: Survey conducted by partner. Survey is informal and has a margin of error of +/- 10% • The USAID intervention is expected to cause 5 more students (for every 100) to stay in school longer 8

  9. IntegrityKey question: Are there mechanisms in place to reduce the possibility that data are manipulated for political or personal gain? L-MEP Liberia - Monitoring & Evaluation Program Issue: Intentional manipulation • Result: Financial sustainability of targeted CSOs improved • Indicator: Dollars of funding raised from local sources per year • Source: Structured interviews with targeted CSOs • When a AO Team member conducted spot checks with the CSOs, she found out that organizations CCC and GGG counted funds from other donors as part of the “locally raised” funds. 9 9

  10. Techniques to Assess Data Quality WHY Goal is to ensure AO team is aware of: Data strengths and weaknesses Extent to which data can be trusted when making management decisions and reporting L-MEP Liberia - Monitoring & Evaluation Program All data reported to Washington must have had a data quality assessment at some time in the three years before submission. ADS 203.3.5.2 10

  11. HOW? Steps to Conduct Assessment Review performance data Examine data collection, maintenance, processing procedures and controls Verify performance data against Agency data quality standards Reliability, precision, timeliness, validity, integrity If data quality limitations are identified, take actions to address them Triangulate; Supplement with data from multiple sources Report the limitations Revise indicator Document the assessment and the limitations in the Performance Indicator Reference Sheet Retain supporting documentation in files Decisions and actions concerning data quality problems Approach for conducting data quality assessment If data will be included in the annual report, disclose the DQA findings in the “data quality limitations” section of the Annual report L-MEP Liberia - Monitoring & Evaluation Program 11

  12. The DQA Tool How organized: follows the life cycle (audit trail)of the data Start with resume: Performance Indicator Reference Sheet. (Skip that part since PIRS was last session) Interview the right people: who knows at each point in the life cycle? Let’s explore the form and try it. Let’s review the protocol. L-MEP Liberia - Monitoring & Evaluation Program 12

  13. Task IX (in pairs) Select a datum from an indicator for which you have data that you know well Pair with another person from a different organization One person interviews the owner of the data Complete the scoring sheet with recommendations in Assessment column Switch roles and repeat 30mins ea. = 1 hr. L-MEP Liberia - Monitoring & Evaluation Program 13

  14. Session 5. Summary Relation of the 7 indicator characteristics to the 5 data quality standards DQA Tool DQA protocol “There are three kinds of lies: lies, damned lies, and statistics.” --Mark Twain L-MEP Liberia - Monitoring & Evaluation Program 14

More Related