MoH, BPS Statistics, World Health Organization

2. DISTRICT HEALTH SYSTEM PERFORMANCE ASSESSMENT IN INDONESIA: results and future use for benchmarking Why assess district health system performance Indonesia? WHO Health System Performance Assessment (HSPA) framework as template. Adaptation of WHO HSPA framework to Indonesia. Results for Indonesia HSPA. Relating inputs to outcomes: measuring efficiency. Possible uses for benchmarking (work in progress).

3. DISTRICT HEALTH SYSTEM PERFORMANCE ASSESSMENT IN INDONESIA: results and future use for benchmarking Why assess district health system performance Indonesia? WHO Health System Performance Assessment (HSPA) framework as template. Adaptation of WHO HSPA framework to Indonesia. Results for Indonesia HSPA. Relating inputs to outcomes: measuring efficiency. Possible uses for benchmarking (work in progress).

4. Decentralization in 2001 redefined role of central MoH and districts: created new challenges vis-�-vis district-centre information flows. MoH currently in national health planning process. Interest in what existing data can say about district performance and what the implications would be for district benchmarking. Who is the target audience for results of health system performance assessment? Why Assess District Health System Performance in Indonesia?

5. Post-crisis economic recovery continues: Country/Region GNI per capita GDP growth (PPP, 2002) (1995-2002) Indonesia $ 2,990 0.5% East Asia & Pacific $ 4,160 5.4% Decentralization: continuing boundary changes; Number of districts before decentralization: 292. After: 440. International focus on millennium development goals (MDGs): large proportion are health-related indicators for MDGs (e.g., skilled-birth attendance, immunization, etc.).

6. National health goals, similar to those identified by WHO HSPA. Relatively low health expenditure per capita Relatively low health expenditure as percent of GDP Country Health expenditure Health percent of GDP (PPP, 2001) Bangladesh $58 3.5% Indonesia $77 2.4% India $ 80 5.1% Vietnam $ 134 5.1% Philippines $ 169 3.3% Thailand $ 254 3.7% Out-of-pocket (OOP) expenditure: 70%; 20% population �insured�. Variety of reforms in last 10 years: to improve staff distribution, insurance coverage, provider performance (quality), etc.

7. DISTRICT HEALTH SYSTEM PERFORMANCE ASSESSMENT IN INDONESIA: results and future use for benchmarking Why assess district health system performance Indonesia? WHO Health System Performance Assessment (HSPA) framework as template. Adaptation of WHO HSPA framework to Indonesia. Results for Indonesia HSPA. Relating inputs to outcomes: measuring efficiency. Possible uses for benchmarking (work in progress).

8. Outcomes: WHO HSPA defined three overall goals of health system: Improving health (level and distribution). Enhancing responsiveness to legitimate non-health expectations of population (level and distribution). Assuring financial risk protection. Health system efficiency: WHO HSPA framework relates health system outcomes to resource inputs: the goal being to identify the maximum achievable outcomes relative to resource inputs. Health system functions: WHO HSPA framework relates variations in efficiency to differences in the way a health system carries out its four core functions: provision, financing, resource generation, and stewardship.

11. Monitor and evaluate attainment of health system outcomes � and the efficiency of the health system � in a way that allows comparison over time and across systems. Build an evidence base on the relationship between the design and organization of the health system and performance, e.g., identification of characteristics of a well-performing health system (determinants of health system performance). Feedback into the policy debate. Empower public with information relevant to their well-being.

12. DISTRICT HEALTH SYSTEM PERFORMANCE ASSESSMENT IN INDONESIA: results and future use for benchmarking Why assess district health system performance Indonesia? WHO Health System Performance Assessment (HSPA) framework as template. Adaptation of WHO HSPA framework to Indonesia. Results for Indonesia HSPA. Relating inputs to outcomes: measuring efficiency. Possible uses for benchmarking (work in progress).

13. WHO HSPA 2000 Framework Indonesia Application Scope International/National District-level Outcomes HALE Life expectancy Financial risk protection Catastrophic expenditure Responsiveness [Responsiveness] Inputs Health expenditure Human resources Education Facilities Income Female education Access OOP expenditure Intermediate Outcomes Coverage indicators Coverage indicators Provider performance Utilization Risk Factors Smoking

14. DISTRICT HEALTH SYSTEM PERFORMANCE ASSESSMENT IN INDONESIA: results and future use for benchmarking Why assess district health system performance Indonesia? WHO Health System Performance Assessment (HSPA) framework as template. Adaptation of WHO HSPA framework to Indonesia. Results for Indonesia HSPA. Relating inputs to outcomes: measuring efficiency. Possible uses for benchmarking (work in progress).

15. CENSUS 2000. SUSENAS and other household surveys. MoH inventories: human resources; facilities. National health accounts. Public health expenditure review. Indonesia Human Development Report 2001, 2004. System-wide perspective: used population-based data where possible; private- and public-sector data where available.

16. WHO HSPA 2000 Framework Indonesia Application Scope International/National District-level Outcomes HALE Life expectancy Financial risk protection Catastrophic expenditure Responsiveness [Responsiveness] Inputs Health expenditure Human resources Education Facilities Income Female education Access OOP expenditure Intermediate Outcomes Coverage indicators Coverage indicators Provider performance Utilization Risk Factors Smoking

21. WHO HSPA 2000 Framework Indonesia Application Scope International/National District-level Outcomes HALE Life expectancy Financial risk protection Catastrophic expenditure Responsiveness [Responsiveness] Inputs Health expenditure Human resources Education Facilities Income Female education Access OOP expenditure Intermediate Outcomes Coverage indicators Coverage indicators Provider performance Utilization Risk Factors Smoking

22. Reduce other basic expenses Push some house-holds into poverty Forgo health services and suffer illness

24. Indicators for the ten provinces with lowest catastrophic expenditures

25. Poverty. If poverty increased by 1%, 27,000 more people would face catastrophic expenditures. Insurance. If insurance coverage increased by 1%, 83,000 more people would be protected from catastrophic expenditures Health Spending. If households spent 1% more of their budget on health, 1.5 million more people would face catastrophic expenditures Predicted effects of changes in poverty, health insurance, and levels of household health spending, on catastrophic expenditure (2001):

27. WHO HSPA 2000 Framework Indonesia Application Scope International/National District-level Outcomes HALE Life expectancy Financial risk protection Catastrophic expenditure Responsiveness [Responsiveness] Inputs Health expenditure Human resources Education Facilities Income Female education Access OOP expenditure Intermediate Outcomes Coverage indicators Coverage indicators Provider performance Utilization Risk Factors Smoking

29. WHO HSPA 2000 Framework Indonesia Application Scope International/National District-level Outcomes HALE Life expectancy Financial risk protection Catastrophic expenditure Responsiveness [Responsiveness] Inputs Health expenditure Human resources Education Facilities Income Female education Access OOP expenditure Intermediate Outcomes Coverage indicators Coverage indicators Provider performance Utilization Risk Factors Smoking

38. WHO HSPA 2000 Framework Indonesia Application Scope International/National District-level Outcomes HALE Life expectancy Financial risk protection Catastrophic expenditure Responsiveness [Responsiveness] Inputs Health expenditure Human resources Education Facilities Income Female education Access OOP expenditure Intermediate Outcomes Coverage indicators Coverage indicators Provider performance Utilization Risk Factors Smoking

42. WHO HSPA 2000 Framework Indonesia Application Scope International/National District-level Outcomes HALE Life expectancy Financial risk protection Catastrophic expenditure Responsiveness [Responsiveness] Inputs Health expenditure Human resources Education Facilities Income Female education Access OOP expenditure Intermediate Outcomes Coverage indicators Coverage indicators Provider performance Utilization Risk Factors Smoking

48. WHO HSPA 2000 Framework Indonesia Application Scope International/National District-level Outcomes HALE Life expectancy Financial risk protection Catastrophic expenditure Responsiveness [Responsiveness] Inputs Health expenditure Human resources Education Facilities Income Female education Access OOP expenditure Intermediate Outcomes Coverage indicators Coverage indicators Provider performance Utilization Risk Factors Smoking

51. WHO HSPA 2000 Framework Indonesia Application Scope International/National District-level Outcomes HALE Life expectancy Financial risk protection Catastrophic expenditure Responsiveness [Responsiveness] Inputs Health expenditure Human resources Education Facilities Income Female education Access OOP expenditure Intermediate Outcomes Coverage indicators Coverage indicators Provider performance Utilization Risk Factors Smoking

53. DISTRICT HEALTH SYSTEM PERFORMANCE ASSESSMENT IN INDONESIA: results and future use for benchmarking Why assess district health system performance Indonesia? WHO Health System Performance Assessment (HSPA) framework as template. Adaptation of WHO HSPA framework to Indonesia. Results for Indonesia HSPA. Relating inputs to outcomes: measuring efficiency. Possible uses for benchmarking (work in progress).

58. DISTRICT HEALTH SYSTEM PERFORMANCE ASSESSMENT IN INDONESIA: results and future use for benchmarking Why assess district health system performance Indonesia? WHO Health System Performance Assessment (HSPA) framework as template. Adaptation of WHO HSPA framework to Indonesia. Results for Indonesia HSPA. Relating inputs to outcomes: measuring efficiency. Possible uses for benchmarking (work in progress).

59. HSPA framework can be used for setting benchmarks. Benchmarks for input indicators could be in terms of targets for health workers, health facilities, educational attainment, etc. Benchmarks may be defined in terms of attainment of health system outcomes (e.g., attainment of life expectancy of x years). Benchmarks for outcomes could be based on attainment of best-performing districts at different input levels.

60. Other examples of benchmarking frameworks. UN Millennium Development Goals (MDGs): Target Indicators Reduce by two-thirds Under-5 mortality rate under-5 mortality rate Infant mortality rate (1990-2015) Proportion of 1-year old children immunized against measles Reduce by three-quarters Maternal mortality ratio the maternal-mortality ratio Proportion of births (1990-2015) attended by skilled personnel UNDP Human Development Index (HDI): Composite index of health, education, and income. No specific targets; can be used to monitor improvements over time.

61. Other examples of benchmarking frameworks. Healthy Indonesia 2010 and Minimum Service Standards: Indicators Target Percent coverage of births attended by trained staff 90% Percent of contraceptive users 70% Percent of villages with universal child immunization 100% Percent coverage for puskesmas visits (HI2010) / Percent coverage of outpatient visits (MSS) 15% Percent of sub-districts free of severe malnutrition 80%

62. What indicators should be chosen? How should targets be set for the chosen indicators? Is measurement of chosen indicator feasible? Is it valid and reliable? Can the indicator be measured reliably over time?

63. Are the indicators policy-reactive? Should benchmarks be chosen for input as well as outcome indicators? Are district-representative data available for the chosen indicators? What are the implications for future survey design? Are resources available for data collection? Statistical capacity for analysis?

64. How to deal with problems relating to data quality and measurement error? Should there be benchmarks for non-health related aspects of health system, e.g., responsiveness, patient satisfaction, waiting time, etc.? Benchmarks related to quality of care? Medical error rates? Compliance with protocols? How many indicators? What is the marginal information content of an indicator?

65. Various frameworks for benchmarking, including WHO HSPA Benchmarks may include input, output, outcome, non health-related aspects of health system, quality of care Input vs Output: Measure Efficiency Indicators selection and target setting Problems related to data availability and quality (HIS)