A Few Thoughts to Start

1. Practical Issues in Implementing Performance-Based Contracting Health System Innovations Workshop Abuja, Jan. 25-29, 2010

2. A Few Thoughts to Start • Don’t Panic: Not everything has to be perfect, be creative! Make new & interesting mistakes – don’t repeat old ones • Be Systematic: Write things down in a contract and contracting manual • The Limits of Planning: Endless planning and analysis can get in the way of action & learning • Humility:a) don’t be too sure of things b) knowledge must be larger than our experience c) give people at local levels sufficient autonomy d) keep learning, evaluating, adapting • :

3. Performance-Based Contracting for Health Services in Developing Countries-A Toolkit

4. Outline of the Toolkit • Summary of the Toolkit – pages 1-8 • What is performance-based contracting? Definitions and Concepts – pages 9-17 • How to Contract? Going through the 7 steps of the contracting cycle – pages 19-66 • Checklist for Contracting – quick summary of tasks in contracting – pages 67-68 • Whether to contract? Review of the global experience with contracting – pages 69-96

5. Outline of the Toolkit • Appendix A: Example of a Contracting Manual including an example of a contract • Appendix B: World Bank procurement approach to contracting for health services • Appendix C: Description of Evaluated Contracting Experiences • Appendix D: TORs for 3rd party evaluation • Appendix E: TORs for contracts involving: (i) PHC delivery; and (ii) HIV prevention for CSWs • WWW.rbfhealth.org - tools and guidelines

6. 1. Dialogue with Stakeholders 2. Define the Services 7. Carry out Bidding Process and Manage the Contracts 3. Design the Monitoring and Evaluation 6. Draft Contract & Bidding Documents 4. Decide how to Select Contractors and Establish Price 5. Arrange for Contract Management The Contracting Cycle: A Systematic Approach (page 20)

7. The 6 most important mistakes • Not clearly defining the objectives of the contract and the indicators of success • Allowing contractors & purchasers to forget the stated objectives & targets • Limiting the managerial autonomy of contractors • Not defining the size & location of each “lot” • Not having a contracting plan: not indicating how contracts will be managed & M&E will be done • Setting prices irrationally and not choosing the best contractors

8. Defining Objectives – More than SMART • Big advantage of contracting is results focus so concentrate on outputs not inputs. • The purchaser should objectively define: • Quantity of services (e.g. % DTP3 coverage, skilled birth attendance) • Technical Quality (national technical guidelines) • Equity (ensuring the poor receive services)

9. Defining Objectives – More than SMART • Need to address the most important challenges • In Ghana 70% of HIV transmitted through FSWs, guess what % of grants went to FSWs? • Focus on a Few!! Indicator inflation if > 10  loss of focus, less data collection • Bias towards outputs & outcomes not inputs & processes e.g. DPT3 coverage better than vaccine availability or “micro-planning” – the exception is in measuring “quality”

10. Defining Objectives – More than SMART • They need to be defined in a measurable way – define numerator & denominator precisely. • Avoid undefined terms like “functional” health facility • Set targets broadly: 20%  58% DPT3 coverage is good even if the target was 60% • Within the “span of control” of the contractor, e.g., measuring availability of staff if contractor not allowed hire, transfer, decide payments to health workers

11. Some Examples of Indicators – What’s Wrong with Them? • % of county hospitals with functioning x-ray machines • Number of health workers receiving appropriate training • % of patients using a PHC Center who are satisfied with the services • Low rate of health workers leaving their positions

12. Defining Objectives – More than SMART • Refer to Tasks 4 and 5 (pages 26-31) • Look at table 3.2 (page 28) • Congratulations!! You’ve avoided the first important mistake

13. The 6 most important mistakes • Not clearly defining the objectives of the contract and the indicators of success • Allowing contractors & purchasers to forget the stated objectives & targets • Limiting the managerial autonomy of contractors • Not defining the size & location of each “lot” • Not having a contracting plan: not indicating how contracts will be managed & M&E will be done • Setting prices irrationally and not choosing the best contractors

14. Ensuring Focus on Outputs • Regular (quarterly) discussion of progress on indicators between purchaser and contractor • Carry out independent M&E • Credible threat of sanctions: • Embarrassment works!! Special meetings • Letters to NGO board • Replacement of key managers • Termination of contract • RBC!!!! (Read the bloody contract)

15. Ensuring Focus on Outputs – Performance Bonuses • Sends signals to contractors about important indicators • Challenges of performance bonuses: • Finding indicators that are important & can be measured reasonably frequently • Amount of bonus – enough to signal but not too expensive – about 10% • What contractor can do with the bonus • Design of bonus: (i) improvement from baseline; (ii) absolute target

16. Design of Bonus System

17. Ensuring Focus on Outputs-Results-Based Financing • “Transfer of money or material goods conditional on taking a measurable health related action or achieving a predetermined performance target.” - Eichler and Levine • Contractors are provided payments based on the amount of services they actually deliver

18. The 6 most important mistakes • Not clearly defining the objectives of the contract and the indicators of success • Allowing contractors & purchasers to forget the stated objectives & targets • Limiting the managerial autonomy of contractors • Not defining the size & location of each “lot” • Not having a contracting plan: not indicating how contracts will be managed & M&E will be done • Setting prices irrationally and not choosing the best contractors

19. Ensuring Managerial Autonomy • Decentralize management to people who are closest to the ground reality - • Purchaser can hold contractors accountable for results when managers have responsibility & autonomy - avoids the “blame game” • Encourages innovation – RBF, sub-centers • Take advantage of private sector’s flexibility – that’s why they’re called Non-Governmental Organizations

20. Impediments to Managerial Autonomy • Telling contractors “how” they should deliver services (define objectives, “what”) • Line item budgets with reimbursement of actual expenditures • Limit flexibility to move money where it’s needed • Encourages micro-management • Increases transaction costs & arguments • Encourages focus on inputs Lump-sum allows proper FM!!!

21. Example of a Line-Item Budget

22. Impediments to Managerial Autonomy • Following Government procedures for staff hiring, firing, transfer, & pay. • Unclear authority of purchaser’s officials • Will try to force certain approaches • Will claim power over more decisions including staff recruitment • Government procures important inputs (allow contractors to do procurement, use private auditors)

23. Drug Availability Index (max = 100) according to who is responsible for drug procurement - Afghanistan

24. A Tale of 2 Countries: Afghanistan & DRC • Both severely affected by conflict • Poor countries with limited infrastructure • Lots of donor money flowing in • Lots of NGOs • Both started contracting with NGOs to deliver health services around 2004 • Used different approaches to contracting

25. Government Controls on Contracted NGOs

26. Change in the utilization rate for curative care (per 100 population per year) during the first year of the contracts 2007 2008 2004 2005 DRC Afghanistan

27. Increasing Managerial Autonomy • See tasks 26 – 29 pages 55-59

28. The 6 most important mistakes • Not clearly defining the objectives of the contract and the indicators of success • Allowing contractors & purchasers to forget the stated objectives & targets • Limiting the managerial autonomy of contractors • Not defining the size & location of each “lot” • Not having a contracting plan: not indicating how contracts will be managed & M&E will be done • Setting prices irrationally and not choosing the best contractors

29. Size of “Lots” – Economies of Scale • Economies of scale in price per beneficiary • Fixed management and admin. costs. • Likely more competition  lower prices • Large packages facilitates contract management • Easier & cheaper to monitor and evaluate contractor performance with fewer lots • Likely to reduce opportunities for corruption • Unscrupulous officials will intimidate small contractors • Easier to avoid “ghost” NGOs

30. Scale of Contracts – Provinces vs. Clusters in Afghanistan • Provinces: 300,000 – 900,000 population • “Clusters” of districts – 100,000 – 180,000 • Issues related to NGO capacity, equity • Bid price of provinces = $4.05 per capita per year vs. $7.80 for clusters, high admin. costs • MOH decided cluster approach was too expensive

31. Size of “Lots” – Arguments for Smaller Scale • Concerns regarding contractor “capacity” • hard to predict & compared to what alternative • Increased diversity – no oligopoly • No disruption to existing providers • But could suffer from “Swiss cheese”

32. Distribution of NGO HCs in Afghanistan un-served

33. Size of “Lots” - Recommendations • Lots should cover at least 0.5 million population • 7-20 contracts • See task 8, pages 34-36

34. The 6 most important mistakes • Not clearly defining the objectives of the contract and the indicators of success • Allowing contractors & purchasers to forget the stated objectives & targets • Limiting the managerial autonomy of contractors • Not defining the size & location of each “lot” • Not having a contracting plan: not indicating how contracts will be managed & M&E will be done • Setting prices irrationally and not choosing the best contractors

35. Use a Contracting Plan • People tend to focus on the contract itself and the recruitment procedures • Fail to systematically address: • How monitoring and evaluation will be carried out (otherwise indicators in contract are meaningless) • How contracts will be managed (client’s activities not described in the contact)

36. Monitoring and Evaluation • Ensure that contracts remain output and outcome focused • Learn lessons and improve performance • Definitions: • “Monitoring” tracking the performance of individual contractors • “Evaluation” tracking the overall progress in service delivery of all contractors (comparison to other service providers)

37. The Process of M&E • Collect baseline data early on & provide to all stakeholders (helps contractors identify important issues) • Recruit 3rd party to help with M&E design & data collection while maintaining government oversight & involvement • Clear responsibility for analysis of M&E data • Ensure there’s data on effectiveness (coverage), equity, quality of care, cost

38. Recommendations • M&E key to achieving good results –worth the investment needed. • Use different sources of data – none are perfect • Make M&E somebody’s job • See tasks 10-15, pages 37-44

39. Contract Management • Often done poorly • Ministries of Health often have limited experience and understanding of contracting • Often not clear who in the MOH is responsible for contract management

40. Major Issues and Challenges in Contract Management • Paying contractors on time • Avoiding corruption • Proper supervision, monitoring & evaluation • Solving problems – many related to relationships • Maintaining government ownership, oversight, & involvement while avoiding micro-management

41. Some Possible General Solutions to Contract Management • Recruiting sufficient number of talented people to manage contracts • Allow sufficient budget • Computerized contract management systems • Provide incentives to contract managers based on results achieved by contractors, timely payment, lack of audit objections, etc.

42. The 6 most important mistakes • Not clearly defining the objectives of the contract and the indicators of success • Allowing contractors & purchasers to forget the stated objectives & targets • Limiting the managerial autonomy of contractors • Not defining the size & location of each “lot” • Not having a contracting plan: not indicating how contracts will be managed & M&E will be done • Setting prices irrationally and not choosing the best contractors

43. Determining the Price of a Contract There are basically 3 options, each with benefits and issues: • Competition at least partly on the basis of price • Negotiation of price with selected bidder • Fixed price where client sets price in advance

44. Competition, at least partly on Price Advantages • lowest price • transparent • encourages innovation • reflects local realities • benefits local NGOs Disadvantages • may take longer • may end up with prices that are inconsistent • may end up with prices that are too low or too high

45. Negotiations with Contractor Advantages • fairly quick • reflects local realities • if purchaser well motivated & savvy can be low cost Disadvantages • not transparent!!! • may end up with prices that are inconsistent • may end up with prices that are too low or too high

46. Fixed Price Advantages • fairly quick • transparent • uniform, hence “fair” (?) Disadvantages • rigid, “one size fits all” • discourages innovation or cost savings • hard to estimate real costs (be humble!)

47. Example of Competition (LCS or QCBS) • Bidders submit technical and financial proposals • technical proposals scored by evaluation committee • financial proposals opened publicly • In LCS, lowest price among technically responsive bidders is selected. • In QCBS technical “score” combined with financial score using a weight (e.g. 80/20)

48. Name of NGO Technical Score (St) Financial Proposal ($) Financial Score Sf =100 x Fm/F Total Score (S) = St x T+ Sf x P A* 60 $2.0 million 100 68.0 B 70 $2.5 million 80 72.0 C 75 $4.0 million 50 70.0 Example of LCS: Minimum Acceptable Technical Score = 60

49. Name of NGO Technical Score (St) Financial Proposal ($) Financial Score Sf =100 x Fm/F Total Score (S) = St x T+ Sf x P A 60 $2.0 million 100 68.0 B* 70 $2.5 million 80 72.0* C 75 $4.0 million 50 70.0 Example of Competition (QCBS) * 2.0m/2.5m x 100=80 and (70 x 0.8) + (80 x 0.2) = 72.0

50. Contractor Selection – Competition vs. Sole Source • Bidders want transparent process with a “level playing field”  competition • Competitive process will generally lead to “best” managers, most innovative ideas, “best” organizations, best prices • Sole source selection is quick but not fair, not transparent, creates resentment, leads to “fat & happy” contractors, limits innovation & creative thinking