By Dr. Ntuli A. Kapologwe ( Director Health, Social Welfare & Nutrition)

Data Collection and Management for Monitoring and Evaluation of Health Services at LGAs level By Dr. Ntuli A. Kapologwe ( Director Health, Social Welfare & Nutrition) & Dr. Anna L. Nswilla (Assisst. Director Health Services) Health Depart. PORALG, Dodoma

Outline • Objectives of the session • M&E tools used at the LGAs level • M&E Framework • Data Sources & Data Registers • Challenges • Way forward

Discussion lies on: • Routine data use for enhanced efficiency, quality and targeting of health services • Innovative strategies for improving quality of data • Synthesis of data from multiple data sources and health information systems for reporting, strategic planning, and continuous quality improvement • New technologies for data collection, management, visualization and dissemination

Objectives of the session • To bring to your attention a few key issues on data collection and management ( Analysis, interpret & Use) requires serious attention by Regional & Districts as it implements the M&E framework of LGAs institutions Dr. Anna Nswilla

M&E tools used at the LGAs level Registers/logbooks • OPD register • IPD register • ANC register • Lab counter book or register • Dispensing form Tally Sheets • OPD tally sheet • IPD tally sheet • ANC tally sheet • Lab tally sheet • Dispensing form Reporting tools • IDSR reporting form • OPD Monthly report • IPD Monthly report • ANC Monthly report • Lab monthly report • ILS monthly report • M&E Database (computerized) • MTUHADHIS2 • eLMS Dr. Anna Nswilla

M&E Tools for • In order to collect the information detailed above, you have data collection and reporting tools that you can use to manage data. • This is a list of the tools used the collection point level at HFs

M & E Strategy • Determine what information is needed to answer the questions. • Generally, there are two types of data that can be collected: • Quantitative data are defined in numeric terms, including percentages, averages and increases. • Quantitative data answer questions such as how many and how much and are gathered via monitoring forms, surveys and other means [ SO CALLED DATA REGISTERS/DATABASES] • Qualitative data can be described in terms of perceptions, feelings, opinions and reasons. • Qualitative data address why and are gathered via focus groups, in-depth interviews and other means.

M & E Strategy • Determine what methods and tools WE must use to gather the information • Think about the kinds of activities you must carry out in order to obtain the information you need. • Process evaluation generally requires methods such as tracking activities against timelines and checklists, and using questionnaires to obtain feedback from participants. • Outcome evaluation employs tools such as Registers ( Foms, Knowledge, Attitudes and Practice (KAP) survey databases and focus groups to gather both quantitative and qualitative data about the effects the strategy has had.

Challenges of the routine system • Clear understanding of indicators by all providers • Incomplete recording done at data collection points (facilities) • Too many reporting forms • Untimely reporting • Data utilization at all levels • Incorrect recording • Lack of feedback mechanisms Dr. Anna Nswilla

M&E Framework • An M &E framework provides guidance for operationalization of the a M&E System. • eg. The M&E Framework for PORALG M&E System guides the collection, analysis and interpretation of monitoring and evaluation data from all implementing agencies and other stakeholders for use in informing key decisions and plans of the whole spectrum of stakeholders.

Definition of Data Source • Data sources are COMPILED tangible sets of information, usually in the form of reports, survey results, monitoring forms from the field, or official government data sets. • Data sources provide the values of the indicators at a specific point in time. • The sources are clearly defined in terms of responsibility for provision, recurrence, and funding source. • At least one data source has to be defined for each indicator. • A data source may provide information for more than one indicator. • Data registers provide the raw data for data sources

Two types of data registers • Data registers are mainly for capturing quantitative data in the M&E system • Quantitative data are defined in numeric terms, including percentages, averages and increases. • Quantitative data answer questions such as how many and how much and are gathered via monitoring forms, surveys and other means [ SO CALLED DATA REGISTERS/DATABASES] • Two types of data registers: • Operational data registers • Survey data registers (or rather survey databases)

Way forward for indicator data element definitions • The definitions must be provided for each data element in a data register • The aim is to convey one consistent meaning for each data element which ensures data quality at the source • PORALG will need to train and get all M&E officers to consider having data element definitions as part of the M&E system. • Ideally all data registers as part of the M&E system should be standardized and as much as possible computerized for easy uploading and communication of the information

Key terms What is data quality? • Data • Indicator • Quality data • Data quality checks Dr. Anna Nswilla

DATA

Indicator • A specific marker or pointer that uses data to show a certain change over an event’s course # of positive blood slide results by month Percent positive for malaria by blood slide # of blood slides done by month Total number of malaria tests done at a facility in a month period, (mRDT and blood slide combined) Dr. Anna Nswilla

Quality data • Data that are reliable and accurately represent the measure they were intended to present • High levels of data quality are achieved when information is valid for the use to which it is applied and when decision makers have confidence in and rely upon the data Dr. Anna Nswilla

Data quality checks • Procedures for verifying that forms, registers and databases are completely and correctly filled at each step of the reporting process. • Examples: • Spot-checks • Cross-verifications Dr. Anna Nswilla

Facility level Why is good quality data important? Basis for planning and developing interventions Allows providers to identify patients/clients in need of services and/or referrals Improves efficiency through administrative organization Inventories resources and determines which supplies and medicines are available and which need to be ordered when Monitors and evaluates quality of care • Basis for planning and developing interventions • Allows providers to identify patients/clients in need of services and/or referrals • Improves efficiency through administrative organization • Inventories resources and determines which supplies and medicines are available and which need to be ordered when

Why is good quality data important? • Many standardized data collection tools collect information that can be used to monitor the quality of care, • However, program staff often do not have the time or capacity to create reports which help them look at the data and understand what it is telling them and how it can be used to improve patient care. • Often, once data reports are generated, the information is not used to enhance patient and program management in health facilities. Dr. Anna Nswilla

Why is good quality data important? • Monitors and evaluates quality of care • Informs acquisition and distribution of resources • Provides evidence for construction and/or expansion of facilities • Explains human resource capabilities and challenges at the facility level • Assists with more precise budgeting • Incorporates DH, HC & Dispensary plans into larger council plans • Assists council authorities in planning interventions and monitoring those activities District level

Regional level Why is good quality…. • Informs • Informs acquisition and distribution of resources • Provides evidence for evaluating administrative performance • Demonstrates trends in calculated indicators used to estimate future changes

Why is good quality…. • Informs policy • Assists in planning and assessing various interventions to make strategic decisions about the improvement of those interventions • Works towards meeting the overall national goal of reducing the burden of diseases in Tanzania • Provides the basis for monitoring the trend of the epidemic National level

Why is good quality…. • Many standardized data collection tools collect information that can be used to monitor the quality of care, however, program staff often do not have the time or capacity to create reports which help them look at the data and understand what it is telling them and how it can be used to improve patient care. • Often, once data reports are generated, the information is not used to enhance patient and program management in health facilities.

When should data quality happen? Data entry (validation checks) Data collection (form completion) Data reporting (audit checks internal and external) All levels where data flows Dr. Anna Nswilla

Dimensions of data quality • Accuracy • Reliability • Completeness • recognizability, • Precision • Timeliness • Integrity • Confidentiality • validity, relevance, accessibility, legality. • challenges of quality assurance problems. • overused: underused:

characteristics of high quality health data • validity, reliability, completeness, recognizability, timeliness, relevance, accessibility, security, legality. • challenges of quality assurance problems. overused: underused:

Accuracy • Also known as validity • Accurate data are considered correct when the data measure what they are intended to measure • Accurate data minimize error (e.g., recording or interviewer bias, transcription error, sampling error) to a point of being negligible Dr. Anna Nswilla

Precision • Data have sufficient detail meaning they have all the parameters and details needed to produce the required information Dr. Anna Nswilla

Completeness • All variables in either reporting or recording tools must be filled • Data represent the complete list of eligible persons or units and not just a fraction of the list • By 7th date of each months – Ministry- OR-TAMISEMI Dr. Anna Nswilla

Timeliness • Data are up-to-date (current) and information is available on time • All reports are submitted to the next level within the recommended timeframe • Timeliness is affected by: (1) the rate at which the program’s information system is updated; (2) the rate of change of actual program activities; and (3) when the information is actually used or required.

Timeliness • Data are up-to-date (current) and information is available on time • All reports are submitted to the next level within the recommended timeframe Due May 7th Dr. Anna Nswilla

Reliability • The data generated by a programme’s information system are based on protocols and procedures that do not change according to who is using them and when or how often they are used • Data are reliable because they are measured and collected consistently Dr. Anna Nswilla

Integrity • Data have integrity when the system used to generate them are protected from deliberate bias or manipulation for political or personal reasons Dr. Anna Nswilla

What contributes to poor data quality? Factors that contribute to poor data quality Common sources of errors Transposition Copying Coding Routing Consistency Range Gaps Calculation • Data entry errors • Inconsistent reporting forms • Missing data • Delayed reporting • Failure to report Dr. Anna Nswilla

Transposition error • When two numbers are switched - usually caused by typing mistakes • Example: 12 is entered as 21 12 Transposition error Dr. Anna Nswilla

Copying error • When a number or letter is copied as the wrong number or letter • Example: 0 entered as the letter O Letter Number O 0 Entered as Dr. Anna Nswilla

Coding error • When the wrong code is entered • Example: Interview subject circled 1 = Yes, but the coder copied 2 = No during coding Entered as 4 during interview Coded as 3 in the dataset Dr. Anna Nswilla

Routing • When a number is placed in the wrong field or in the wrong order • Example: Gender entered into the age category) Gender erroneously entered into the age category

Consistency • When two or more responses on the same questionnaire are contradictory • Example: Birth date and age; name and gender Mary erroneously entered as a male Dr. Anna Nswilla

Range • When a number lies outside the range of probable or possible values • Example: Age = 151 years Weight erroneously entered as 400kg Dr. Anna Nswilla

Gaps When data are not filled in Unique ID is missing

Calculation • When calculations are not done correctly • Example: 3+1 = 5 340 = 110 + 230 Total males and females added erroneously

Quality Health Data • Data quality is defined as “the totality of features and characteristics of a data set that bear on its ability to satisfy the needs that result from the intended use of the data.” • High quality data effectively satisfies its intended use in decision making and planning. • Healthcare professionals must be able to rely on the information presented. • All users of health information must be able to interpret the data that are presented in the health record.

Quality Health Data… • While many organizations boast of having good data or • improving the quality of their data, the real challenge is defining what those qualities represent. ... • in another industry (which is to say, inaccurate data in healthcare could have more serious consequences) and, • therefore, justifiably worth higher levels of investment.

Performance Indicators Indicators from DHSI2 Verifier IAG Report Findings – affected disbursement • HBF Scorecard Performance indicators – Disbursement Link Indicators DLI 12 PI HBF and DLI3 for RBF

Thank you for listening

By Dr. Ntuli A. Kapologwe ( Director Health, Social Welfare & Nutrition)