Inter-Agency Real Time Evaluation – Briefing Response to the 2009 Pakistan IDP crisis

1. Inter-Agency Real Time Evaluation – BriefingResponse to the 2009 Pakistan IDP crisis

2. Structure of this presentation The evaluation and the methods used The evaluation criteria Findings by criteria Conclusions Recommendations Caveat: This presentation presents a preliminary view of the conclusions drawn by the team. Additional information during the comments phase may lead to changes in the conclusions.

3. This evaluation This evaluations is the eight Inter-Agency Real-Time Evaluation (IARTE) of the current series. It is the third IARTE in Pakistan and is the second IARTE of a response in a complex emergency. The evaluation is managed by OCHA on behalf of the Inter-Agency Standing Committee (IASC). It is funded by five agencies (DFID, OCHA, UNDP, UNHCR and UNICEF). The evaluation has been conducted by a team of 3 consultants working for Dara, an independent organisation specialising in evaluation. It is the third IARTE that Dara has worked on.

4. Methods

5. The team used a mix of methods including: Group interviews with 846 affected persons. Direct observation on visits to 12 different locations. The team visited locations in Lower and Upper Swat, Kohat, and Hangu, as well as Jalozai and Peshawar. Meetings with over 140 persons from the Humanitarian Community, Government, Donors, and the Military. Document research Methods

6. Evidence table The team collected 1,245 pieces of evidence as bullet-point notes totalling some 26,000 words. The evidence for the different criteria was distributed as follows:

7. Triangulation

8. Effectiveness (of response given as compared to objectives and targets defined) Consistency (of response given as compared to standards adhered to), Relevance (of the response given as compared to needs) Timeliness (time elapsed between displacement, assessments and response) Connectedness  (sustainability as achieved through ensuring that activities are planned in support to pre-existing response structures and capacities) Evaluation criteria:

9. The Context

10. In any complex emergencies the “forces” that bring about emergency are human and therefore the social, cultural, economic, and political context are key: Pakistan today is a complex environment and the context is the key to understanding the IDP crisis. Complicating factors include: chronic poverty, fragmentation of the state, the colonial legacy, inequality, military dominance, poor governance, and regional and international dimensions. The affected area already suffers from some chronic problems around basic services. Women are marginalised in Pakistan. The Context

11. We looked at the following topics under the effectiveness criterion Achievements Security constraints Access constraints Coordination Funding and resourcing General learning for the future Reviews and evaluation Effectiveness

12. Findings on effectiveness 1 0f 3 There were significant achievements in first phase of response in Swat – inc. low mortality and morbidity in a very large-scale emergency The contribution of the local community played a very large part in the effectiveness of the response Security is a serious and well founded concern and security still constrains and impacts the response. The HCT is not providing strong leadership for the response. There is a gap between needs and funding for 2010 that is curtailing the ability to meet needs.

13. Findings on effectiveness 2 of 3 Assistance was not always a good match with needs Assistance was not proportional, camp needs better served than hosted, relief better than recovery, focus in some geographic areas, host families needs rarely met, and stayee needs not met. Clusters performance has varied – usual problems. Coordination mechanisms took too long to set up and meetings more for information exchange rather than for decision making Major problems with coordination within Government both between and within tiers of government.

14. Findings on effectiveness 3 of 3 The needs were non-linear with fresh displacement in some areas at the same time as return in others. Key lessons around: Adapting the response to the needs; The need for better levels of preparedness; The need to establish coordination mechanisms faster; The advantages of rapid registration; The need for effective agreed tools (joint assessments and common reporting system); The need to give greater emphasis to humanitarian diplomacy.

15. Consistency Under consistency we address both the question of coherence to both humanitarian principles, and to operational standards for providing assistance. Neutrality, Impartiality, and Independence Relations with the Government Relations with the military Attention to gender issues Monitoring systems Reviews and evaluations Learning based changes

16. Findings on consistency 1 of 2 Response was not principled. The extent of the trade off between principles and access varied by actor. The UN strongly supported government. Military are party to the conflict for which they control humanitarian access. Military have a key role in responding to emergencies in Pakistan. Easier for NGOs to coordinate with military than with civil administration. Government critical of NGO interventions and the international response, but response would have been far weaker without international support

17. Findings on consistency 2 of 2 Significant encroachment on humanitarian space. Decisions concerning who got assistance sometimes influenced by political and military considerations. Little understanding of humanitarian principles. Little use of humanitarian diplomacy. Some attention to gender issues in 2010 but women's needs were often not addressed Monitoring problems compounded by problems of access and security. Relatively little review of what was done. Some learning, but the process seems to have been haphazard and ad hoc

18. Relevance Relevance addresses the extent to which the assistance provided was relevant or appropriate to the needs of the population. We examined the following topics: Needs assessments undertaken Camps/ Host/ Other Consultation of affected population Gaps between Needs and Assistance Targeting (including distribution lists and registration criteria)

19. Findings on relevance 1 of 2 Registration criteria flawed - major inclusion and exclusion errors both in status and in vulnerability. Response was category based rather than status or needs based Relatively little needs assessment was undertaken. Few joint needs assessments were carried out. Most were carried out for individual agency needs Assistance provide by host families very relevant Group with the most assistance Group with the least assistance

20. Findings on relevance 2 of 2 The affected population was usually not effectively consulted. When consulted, was the leaders that were consulted rather than the general population. Women were generally not consulted In several cases assistance packages, especially in-kind assistance, was not very appropriate. Failure to assess needs led to waste. Some needs in the area are chronic (sanitation, nutrition, etc) and not specific to the displacement crisis. Rent is a very significant cost for many families New Vulnerability Approach welcome.

21. Timeliness Timeliness is a key aspect of the effectiveness of the emergency response. We looked at two issues under timeliness: Bureaucratic constraints (both internal and external) Delays in response

22. Findings on timeliness Local response critical in early stages – it was fast and timely Biggest delays are in recovery PHRP delayed by Govt NOC system is a serious concern Bureaucratic constraints used to control NGOs Access issues significantly constrain a timely response

23. Connectedness Connectedness refers to the extent to which short term interventions tie into the longer-term needs of the community. The topics we have looked at under this heading include: Connectedness Recovery constraints Changes from 2009 to 2010

24. Findings on connectedness Most of the response focus has been on relief rather than on recovery, rehabilitation and development. Early recovery strongly constrained by funding and recovery needs are not being met. IDPs interviewed generally prefer cash assistance to in-kind assistance to recover. Cash also allows to reactivate the local market, but need appropriate controls. People will not return unless they feel secure. Major constraint for recovery is direct damage to infrastructure and livelihoods. Biggest changes since 2009 are the scale of movements, donor interest and level of funding, phase of displacement cycle, and geographic focus.

25. Conclusions

26. Conclusions 1 of 4 Overall, this was a humanitarian response set against the backdrop of a very complex environment, where a government mobilised the military to reassert control of the national territory. Given that the primary driver for the military operation was the re-establishment of sovereignty, and that underlying causes of the security problems included international factors, it was not surprising that the Government constrained the response of international humanitarian organisations. The unprecedented displacement in terms of the speed and scale led to prompt action by the local population, the Government of Pakistan and the humanitarian community.

27. The response was effective, preventing large scale death and suffering. While the most important element initially was the support of the local community and of existing infrastructure, the national community also made a significant contribution, as did the international community. However, the response was a set-back for a principled and needs based approach, as different actors balanced the principles of independence, neutrality, and impartiality against the humanitarian imperative. The humanitarian community did not use all of the levers available to it to push for humanitarian space. Conclusions 2 of 4

28. The effectiveness of the international contribution to the response was constrained not only by security, but also by access and bureaucratic controls intended to limit the role of the international community. The action of the humanitarian community has also been constrained by the lack of clear planning and effective coordination within the government. The NOC system as currently administered is a major obstacle to effective humanitarian action. Despite the overall effectiveness of the response there were significant gaps in the coverage of the affected population, sectors, and geography. Conclusions 3 of 4

29. The response highlighted the fact that the international community still has not resolved the issues of coordination, with many of the same problems seen in previous responses still present. The lack of funding for recovery threatens the sustainability of return and increases the risk of future displacement. Despite the problems noted, it was an effective response with some innovative features that is now heading the right way with the VA pilot. The humanitarian community in Pakistan has the capacity to assist the affected population if resources are available. Conclusions 4 of 4

30. Recommendations

31. Recommendations – Stronger HCT The ERC should immediately introduce a requirement that, except in exceptional circumstances, the heads of the three main UN humanitarian agencies (Unicef, WFP, and UNHCR- where present) should attend the regular meeting of the HCT. The ERC should also require HCs to provide each quarter a summary of HCT attendance by heads of agencies. The chair of the HCT meeting should with immediate effect for all decisions, establish who exactly is responsible for implementing the decision, who should report back on it, and when progress reports should be made. Progress against implementation should be reviewed at subsequent meetings.

32. The Humanitarian Country Team should develop an active strategy of humanitarian diplomacy to work for a more principled approach and less constrained humanitarian space in Pakistan, including putting the issue on the agenda for donors. The humanitarian community in Pakistan should reassess all existing programmes, both in terms of the appropriateness of the assistance that they are providing and whom they are targeting. Recommendations: Advocacy

33. The humanitarian country team should roll-out the vulnerability assessment model for the whole affected area and base assistance on vulnerability rather than just on registration. The humanitarian country team should begin a process to map needs in a consistent way across the affected area. Such a mapping may require that the HCT commission a detailed survey of the affected population to discover what types of assistance has been most useful, and what types of assistance are still needed. Recommendations: Needs-Based

34. The government should introduce a ‘presumed to comply’ rule for No Objection Certificates, where, unless NOCs are formally refused, they are deemed to come into effect within seven days from the submission of the application. The humanitarian country team should advocate with donors to adequately fund recovery in the short to intermediate term while waiting for the medium to long term mechanisms to come into play. Recommendations: Other

35. johncosgrave@gmail.com rpolastro@daraint.org Please send your comments to: