Using indicators to measure country pharmaceutical situations

1. Using indicators to measure country pharmaceutical situations Dr. Edelisa D. Carandang Medicines Policy & Supply Management Department of Technical Cooperation for Essential Drugs and Traditional Medicines (TCM) World Health Organization, Geneva Sept 2006

2. Using indicators to measure country pharmaceutical situations • Concepts on pharmaceutical monitoring and assessment • The WHO process on assessing and monitoring pharmaceutical situation • Snapshots of current global pharmaceutical situations using Level I and Level II indicator survey- the fact book • New approaches in access indicators

3. Pharmaceutical monitoring and assessment Monitoring • Review of the progress toward completing program activities and achieving objectives • Allows for corrective action during program implementation • Focus on inputs and outputs -integral part of management. • Common methods in fully developed monitoring systems • Supervisory visits • Routine reporting of selected data • Sentinel sites • Special studies

4. Pharmaceutical monitoring and assessment Assessment • Discussed along with monitoring as part of overall evaluation • Analysis of progress and meeting objectives • Types of evaluations • Needs assessment (situation analysis, • Formative evaluation (midterm review) • Summative evaluation (final evaluation) • Field surveys using standard pharmaceutical assessment indicators & ongoing monitoring system, document review • Strategies for monitoring and assessment developed in parallel for comprehensive unified strategy

5. Principles in designing monitoring and evaluation system • Focus on key monitoring questions and indicators • Minimum data collection • Develop practical procedures • Consider comparisons between facilities and performance over time. • Plan how information will be available for timely feedback and follow-up action • Phase the design and implementation • Design based on the current system

6. Who can use the results from assessment and monitoring? • Countries - focus action, prioritize, measure achievement • National policy-makers • synchronise policies • data and information to donors and other governmental agencies • International agencies • to assess the structure and capability of countries, assess the progress, accomplishment and impact of aid • Professional groups, NGOs and academia • to focus advocacy activities and information campaigns • Health facilities to be aware of institutional problems & improve situations

7. Pharmaceutical indicators • Variables that measure situations and change (numerical, binomial) • Useful tools to track the performance of particular aspects or activities of the pharmaceutical system • Linked to an important input, process, or outcome • Well-established indicators can be adapted/ modified to reflect the realities • Field test

8. Summarizing indicator measures • Percentage: yes or no over total • Measures of central tendency • Mean: average value, sensitive to outliers, weighed toward skewed value, best summary of normally distributed values • Median: middle value, resistant to outliers, good summary of any distribution • Equivalent if data are normally distributed • Measure of variation • 25th and 75th percentiles: boundaries of middle half of values, good summary of the overall spread of values, better summary of skewed data

9. Indicator measure: Ideal/logical values • Ideal value • 100%-adequate labelling, meds dispensed, adherence to STG, availability of medicines, generic, adequacy of storage • 0 days- of stock out • Set by countries • % affordability • Logical value (% use of antibiotics, %injection, ave drug) • complex • can be calculated empirically

10. Indicator measure: group norm • Easy for region/facilities to relate to peers • Norms may be wrong

11. Ministry Target = 90% Why is it important to use indicators:(1)Setting target

12. Why is it important to use indicators:(2) For comparison

13. Why is it important to use indicators: (3) Seeing trends overtime

14. Background on WHO work on indicators and pharmaceutical assessment &monitoring • Indicator and monitoring tools • Indicator for Monitoring National Drug Policy (94 & 99 Ed • WHO Operational package for Monitoring & Assessing Country Pharmaceutical Situation • Publications • 1988 World Drug Situation • 2003 World Medicines Situation • 2006 Using indicators to measure country pharmaceutical situation: Fact book on WHO Level I and Level II monitoring indicators • WHO medicines data base

15. WHO Evidence-Based Planning and Interventions Guiding Country Works in Medicines Indicator-based tools to evaluate structures, processes, outcomes of in countries Develop implementation plans and identify strategies & interventions based on data/information on: availability, affordability, pricing, drug use and regulatory profile, TRIPS, drug management situation. Support implementation of activities and advise in the execution of work plans

16. Why WHO is monitoring and assess pharmaceutical situation using indicators? Objectives • prioritized pharmaceutical policy objectives based on evidence • determine effects, impact of policy implementation over time • establish situation evidence for advocacy Indicators for evidenced based policy • Provides evidence for planning, prioritising and identifying interventions • Systematic data gathering • Allows comparisons & set target • Measuring trends 1. Assess and Monitor Access & rational use of quality medicines 3. Implement 2. Plan

17. WHO hierarchical approach to monitoring and assessing pharmaceutical situations • Level I • Questionnaire/rapid assessment/checklist • Arrays achievement & weaknessess, illustrate sectoral approaches • Level II • Comprehensive monitoring of pharmaceutical strategy outcome and impact • Measures attainment of objectives • Level II • More detailed îndicators for monitoring and evaluating specifc areas/components Level I Core structure & process indicators Questionnaire (Health Officials) Systematic survey Level II Core outcome/impact indicators & household survey • Level III • Indicator tools for specific components • of the pharmaceutical sector • Pricing ● Traditional medicine • Paediatrics ● Assessing regulatory capacity • TRIPS●Supply & Procurement

18. Level I indicators: structure and process indicators • Regular survey questionnaire • Advantages • Cheap way to get information across countries • Can be done repeatedly/regular period • Automated questionnaire and data encoding processing • Limitations • Validation of data • Accuracy of responses • Missing data

19. Level II indicators: WHO Operational Package for Monitoring and Assessing county Pharmaceutical Situations Procedure and guidelines • Administrative preparation: • Coordinating with WHO, ministry/department of health, public health facilities, private drug outlets, warehouses • Making logistic arrangements and budget allocations • Technical requirements: • Selecting geographic areas and facilities • Identifying country-specific items of the survey forms, e.g. key basket of medicines, treatment guidelines, etc. • Training data collectors to carry out the survey and use the survey and summary forms • Analyzing and computing the data • Preparing a report and using result

20. Indicators in the Level II survey • 15 survey forms-public health facilities, public pharmacy/dispensary, private pharmacy, warehouses • indicators • on availability, stock out, record keeping and expiry of key drugs • conservation conditions and handling of medicines • affordability (child and adult moderate pneumonia and option for other disease condition • drug prescribing, dispensing,patient knowledge

21. Sampling for systematic survey • Follow specific procedures • to minimize selection bias • study population is representative of the reference population • A balance between what is desirable and what is feasible- smallest one with a degree of precision

22. Training data collector for the survey • Important for consistency in data gathering process, validity & accuracy of data • Who can be trained? • Physicians, nurses, pharmacists or paramedical staff • Health ministry/department staff and temporary employees (health related background and experience) • data collectors from different parts of the country (language differences) • Fiji -used students from B.Pharm Y3/Y4,Improve FSm/MoH relations • Solomon Islands-pharmacy officers and pharmacists who are not from their own province

23. Level II Indicator (advantages & limitations) Advantages • Practical survey based on small samples • Covers impact and outcome in one comprehensive survey • Efficient in terms of resource requirement owing to standardized process of training, data gathering and time allocation Limitations • Requires more resources for training and conducting systematic survey • Limitation in terms number of countries that can be covered

24. Level III Indicators • Systematic survey and monitoring • Drug price survey and monitoring • Rapid assessment • Global survey on Paediatrics medicines • Questionnaire on public sector medicines procurement and supply management systems in countries • Assessment of regulatory capacity

25. Fact book on Level I & Level II indicator result • Product of years’ of work on developing and improving data-gathering tools, systematic collection of information • Overview of pharmaceutical sector components current status & impact • Countries grouped low, middle or high-income. • Data and information in tables and graphs • current situation – 2003 Level I survey • progress - comparing Level I 1999 and 2003 surveys • impact and outcome measures - Level I & Level II. • Country data in CD ROM

26. Percentage of Countries with Official National Medicines Policy (NMP) in 1999 and 2003

27. Key pharmaceutical sector legislation

28. Generic prescribing and substitution regulations in 1999 and 2003 Generic Prescribing

29. Samples Collected for Regulatory Purposes, Tested, and Failing Tests in Previous Year

30. Storage and handling conditions Storage and handling conditions in public pharmacies and warehouses

31. Countries with insurance that covers medicines Public insurance Private insurance

32. Medicines Pricing Policies

33. Proportion of prescribed medicines dispensed % medicines dispensed

34. Affordability of pneumonia treatment in children

35. Rational Use of Medicines : Specific Conditions

36. Measuring access to essential medicines *Estimate

37. Measuring access to essential medicines ( Household Survey) • Level I and Level II surveys do not measure access from the patient/consumer perspective. • Only household surveys can provide population-based information about how pharmaceutical policies affect the well-being of individuals. • Household surveys were tested on a pilot basis • With Level II surveys to maximize resources

38. 1. Rational selection 3. Sustainable financing 2. ACCESS TO ESSENTIAL MEDICINES 4. Reliable systems 5.Affordable prices Measuring access (Composite indicators) • Measure policy structures supporting access preconditions) • Steps used in the current process Operational • Obtain complete data for candidate variables • Validate data accuracy Statistical • Formal statistical analysis to obtain appropriate factor weights • Validate relationship between Level I structure and process and Level II outcomes indicators

39. Implementation • Develop and execute action plan based on available resources • Prioritize and implement strategies • Formulation • Identify problems • Define objectives • Develop strategies • Monitoring & evaluation • Develop system • Identify tools • Use results National drug policy process

40. Improved drug supply management: Guide selection, procurement, distribution, use Selection Monitoring Monitoring • Management • Support • Organization • Financing • Info. Mangnt. • HR Use Procurement/ Monitoring Monitoring Distribution Medicine Supply Management

41. The way forward on country monitoring • Evidence through systematic but feasible data collection process is necessary in policy making and activity implementation • Should demonstrate that in the long run regular monitoring is not difficult and can be done in a cost efficient manner • Portion of country support budget and project grants should be allotted to monitoring and evaluation using indicators • Information and data sharing is important

42. THANK YOU