Concurrent Evaluation of National Rural Health Mission

1. Status – October, 2008 Concurrent Evaluation of National Rural Health Mission

2. Objectives of the Concurrent Evaluation To evaluate the impact of NRHM interventions, by Agencies outside the Government, and make recommendations on: • The institutional mechanisms and monitoring systems that have been put in place. • Programme designs for consistency, feasibility and effectiveness with respect to goals of NRHM. • Operationalisation of the programme elements of NRHM, identifying managerial and design constraints. • Capacities available and adequacy building efforts. • Programme outcomes with the respect to the phase of the programme in each State.

3. Scope and Extent • By Independent Agencies including PRCs • To cover all Districts within 2 to 2.5 years in Rounds • First Round to cover over 220 Districts from all States/UTs (35% of Districts) • Revisit after completion of Rounds

4. Agencies • Field Agencies • one for each State/Sub-State (UP, MP) • 5 Zonal Agencies • one for 5-6 States • National - IIPS

5. Selection of Agencies • Grants-In-Aid Committee constituted • Included MoSPI, MoRD • Evolved Detailed Criteria for Technical Evaluation • Sub-committees constituted for • Evaluation of Technical Proposals • Evaluation of Financial Proposals • Quality-cum-Cost scoring • 70% weight to technical proposal • 30% weight to financial proposal

6. Roles & Responsibilities: FIELD AGENCY • Desk review • Consultation with key stakeholders • Undertake evaluation in accordance to methodology • Collection, collation & analysis of data using recommended tools • Provide data in prescribed format to Zonal agency • Coordinate with Zonal agency & prepare district & state reports • Conduct Field work in 3 months; final report within one month thereafter

7. Roles & Responsibilities: ZONAL AGENCY • Training and Coordination with Field Agencies on the survey • Support Field Agencies in data validation, compilation, data analysis, report generation • Cross/back check data of Field Agencies • Recommend payments • Coordinate preparation of State/District Reports • Ensure final report from field agencies within 4 months

8. Roles & Responsibilities: NATIONAL AGENCY • Overall monitoring and coordination for Evaluation Work including • field work • data flow • report generation • District/ State/ National reports • Thematic Evaluation • Training of Zonal /Field Agencies • Monitoring the quality assurance of survey results • Develop data entry package, tabulation software and provide training on the software for use by all Zonal and Field Agencies • Legal support for contractual commitments and obligations • Evolving framework for performance appraisal of agencies

9. FINANCIALS • Total cost to be inclusive of: • Hiring of staff including salary and wages • Travel (hiring taxi/renting car mandatory) • Printing of schedules, reports • Photographs • Data collection, cleaning and tabulation • Report writing • Training • Office overheads • Any other incidental charges • No additional cost will be borne by GoI

10. DELIVERABLES • Inception report: within 15 days of 1st installment • Baseline report- Secondary data -Interaction with key stakeholders • Mid project progress report • End report • Case studies and Observation reports • Photographs • Hard and scanned copies of interview schedule • Presentations • Final Report

11. SAMPLING DESIGN Schedules to be filled from: • State HQ • District HQ and Hospitals • Two (2) CHCs in each district • Four (4) PHCs (2 PHCs per CHC) • Twelve (12) sub-centers (3 sub-centers per PHC) • Twelve (12) gram panchayats • 24 ASHAs • 1,200 households

12. Coordination mechanism

13. Process of Selection

14. State-wise Distribution of proposals of Field Agencies

15. Overall Findings ...(1)

16. Overall Findings ...(2) • No bids in 4 States/UTs • Manipur, Meghalaya, Mizoram, Nagaland, • D&N Haveli • Single Bids: • Field – A&N Isles • Zonal – South, North East • Financial Bids have wide variation

17. Overall Findings ...(3) Financial Bids’ variation

18. Post Selection • Concerns • Financial Viability of Agencies • Capability to Deliver as per Schedule • Steps taken • Elaborate MoU for Field, Zonal and National agencies • Bank Guarantee • No Conviction, No Blacklisting undertaking • Key Personnel undertaking • Audited Financial Statement s • Bank Statements • Future Steps • Higher cut off for technicals • Performance in Round I to be given weightage • Rating of Agencies

19. Present Status...(1) • Finalised and Dispatched • Memorandum of Agreement for Agencies • TORs for IIPS as a National Agency • TORs for Supplementary Round • Manipur, Meghalaya, Mizoram, Nagaland, D&N Haveli • Award Letters issued to selected agencies • 32 - Field Agencies • 5 - Zonal Agencies

20. Present Status...(2) • Zonal/Field Agencies • Acceptances being received • Documentation to be vetted • Payment of Advance (40%) • National Nodal Agency (IIPS) • To send acceptance and Financial Costs • Organise TOTs for Zonal/Field Agencies • Tabulation Plan, State/District Reports

21. Present Status...(3) • Supplementary Round • Field Agencies to send proposals • Technical, Financial Evaluation, QCBS etc • Award Letters, documentation etc • Round-II preparations • Repeat of cycle of EOI, RFP, Short-listing etc

22. Concerns • Fresh vs Seasoned Agencies • Increase market size, create capacity in market • Massive coordination activity • Time-lines, contingencies, natural calamities • Involvement of ROs and FSUs of CBHI in coordination

23. Over to HMIS …