IASC Inter – Agency Real Time Evaluations (IA RTEs)

1. IASC Inter – Agency Real Time Evaluations(IA RTEs)

2. Outline • What are IA RTEs? • What are they good for? • Where do IA RTEs come from? • How do IA RTEs work? • In which ways are IA RTEs different from standard evaluations?

3. What are IA RTEs?

4. Definition • IA RTEs are ‘Inter-Agency (IA)’ in that theRTEs are instigated, managed and used by a variety of international organizations • The term ‘Real-Time (RT)’ refers to theearly implementation stages of a humanitarian emergency operation,to the rapid feedback of IA RTE findings at the field level (within the two first month)and consists of remote monitoring and the IA RTE mission. • ‘Evaluation’ (E), as applied in the term IA RTE, describes aparticipatory review process. IA RTEs differ from other humanitarian evaluations regarding speed, coverage, methods, and outputs.

5. What are IA RTEs good for ?

6. Purpose IA RTEs seek to: • identify gaps, access constraints to the “humanitarian space”, • assess the relevance, quality and timeliness of the response, • unlock inter-stakeholder coordination problems or operational bottlenecks, • propose appropriate strategic re-orientation • provide real time learningto the Humanitarian Country Team. • identify strengths and weaknesses of the responsethrough the assessment of the response to the affected population. • The primary end users of IA RTEs are in-country agencies and staff engaged in the humanitarian response

7. SUM UP • IASC mandated evaluation tool that is managed and used by a variety of international agencies for real time learning • Focus is on inter-agency coordination and management. • Carried out at the early implementation stages of a humanitarian operation • Rapid feed back of findings at the field levels for immediate use • Seek to identify gaps, access constraints & potential threats and assesses the quality of the response

8. Where do IA RTEs come from?

10. Towards IA RTEs • Humanitarian Response Review • Humanitarian Reform • Clusters and CERF • RTE

11. Humanitarian Response Review

12. Results of the HRR (excerpt from recommendations): The IASC and the donors should: establish a limited set of benchmarks (and indicators) to be implemented in the first period of a new emergency up to a maximum of 3 months and addressing in priority access and coverage of population in need identification of responsibilities in delivery of assistance and in coordination resources mobilization (human, assets, financial) The IASC should: identify and assign lead organizations with responsibility at sectoral level, especially in relation to IDP protection and care and develop a cluster approach in all priority sectors. (Action: ERC/IASC – immediately)

13. Humanitarian Reform 2005 / 2006 Cluster Evaluations IA RTEs

14. Inter-Agency Standing Committee (IASC) • Unique inter-agency forum for coordination, policy development and decision-making involving the key UN and non-UN humanitarian partners. • Established in June 1992 in response to United Nations General Assembly Resolution 46/182 on the strengthening of humanitarian assistance. • General Assembly Resolution 48.57 affirmed its role as the primary mechanism for inter-agency coordination of humanitarian assistance. • Develops humanitarian policies, agrees on a clear division of responsibility for the various aspects of humanitarian assistance, identifies and addresses gaps in response, and advocates for effective application of humanitarian principles.

15. IASC WG requests IA RTE Pilot Phase IASC WG requests IA RTEs in 2007: • At its 64th meeting held in March 2007, the IASC Working Group endorsed a one-year pilot to inform the developing and testing of approaches to inter agency real time evaluation (IA RTE), with the objective of eventually establishing a regular standby mechanism. At this time, the IASC also endorsed nine tenets / principles of IA RTE and committed to ensuring agency-specific and IASC-wide management response and action to the recommendations arising out of future IA RTEs.

16. Inter-Agency RTE should be used to address broader, cross-cutting and sector wideconcerns. The primary purpose of the IA-RTE is to support field management decision-making; a secondary purpose is to provide information to senior managers in agencies to help better understand and support the programmes they are in charge of directing. The adopted approach should be light, flexible and responsive to the needs of field actors and the changing nature of the emergency response. There needs to be a consistent and timely trigger for IA-RTEs – ideally IA-RTEs should already be indicated in a flash appeal/CAP. IASC WG formulates 9 Basic Principles

17. IA-RTEs should not replace agency-specific RTEs - the target audience is therefore the broader humanitarian community and not a single agency. There needs to be a strong focus on the affected population which must inform the method of the IA-RTE. IA-RTE teams should be lean and reflect a mix of agency expertise, methodological skills and sectoral expertise as is appropriate for the emergency. IA-RTE are management intensive and need a full time lead management agency, supported by an inter-agency management group IA-RTE need toresult in immediate use and action. This also requires a strongcommitment from all participating agencies from field upwards to headquarters to respond to and take seriously the recommendations made by the team. IASC WG formulates 9 Basic Principles

18. 74th IASC WG agreed to move beyond the pilot phase to regular implementation and requested to develop a set of operating procedures: Criteria for triggering IA-RTE A Clear Purpose Statement for IA RTE A standard ToR & methodlogies IASC WG request to move from pilot phase to regular implementation

19. IASC IA RTE Interest Group develops an IA RTE Procedures and Methods Guide IA RTE Procedures and Methods Guide is endorsded by the IASC during its WG meeting in July 2010! 

20. How do IA RTEs work?

21. How do IA RTEs work? • Who are the main Stakeholders in IA RTEs? • How and when to trigger an IA RTE? • What is the Evaluation Framework for IA RTEs?

22. IA RTE Stakeholders

24. Criteria Lists for Triggering IA RTE • Essential IA RTE TriggerCriteria(Automatic) • > 1 million people affected? • AND • Flash Appeal > than $50 million or CAP >than $350 million? • Desirable IA RTE Trigger Criteria • (By request) • IASC WG, SG members, ERC, RC/HC or a majority of HCT members requested IA RTE • Inter-agency coordination & management to be examined cover a specific period & a defined geographical area? • Opportunities for RTE learning? •  Capacity exists to conduct an IA RTE?

25. Sheet I: Automatic Triggering IA RTE: Decision Making Process Who and When 2. OCHA Focal Point (FP): reviews the IA RTE Trigger Criteria TD: +2 3. Does emergency meet automatic trigger criteria? More than 1 million people affected? AND Flash Appeal is > $50 MM ? or CAP Appeal > $ 350 MM? TD: +2 NO YES • 4. OCHA Focal Point: • informs Support Group (SG) of IA RTE launch • requests $ 80,000-120,000 in Flash Appeal • asks SG to establish an ad hoc Management Group (MG), & optional Advisory Group (AG) • posts criteria review & information on IA RTE web • along with MG, informs ERC of IA RTE • TD: +2-7 • 4. OCHA Focal Point: • Posts results of criteria review on IA RTE website • IA RTE still might be triggered by request if it fulfils Desirable IA RTE Trigger Criteria (Consult Sheet 1) • TD +7 5. ERC informs HC/HCT to elicit feedback & response TD: + 7 12. Consultants begin Mission to Country—Interview affected populations and actors TD: +52 6. Practical arrangements by MG (logistics, travel, arranging optional AG etc) TD: + duration of IA RTE 13. End of Mission Debriefing in country TD: +73 7. FP requests SG members begin remote monitoring and data collection TD: 0 until consultants deployed 14. Consultants draft report, additional debriefings to MG TD: +87 8. FP//MG draft ToR, EoI, & publishes EoI on Relief Web & ALNAP TD +8-9 15. HC/HCT, Advisory Group, MG, SG, interviewees comment on report TD: +94 9. FP /MG hires Consultant (s), contracts signed TD: +39 16. Final Report Produced TD: +101 10. Consultants receive remotely monitored data and start desk review TD: +45 17. MG: Approves Report Report is shared with SG TD: +108 11. Briefing between consultants & MG face-to-face or via phone TD: +50 18. Report for follow up to HC via ERC TD: +109 KEY ERC: Emergency Relief CoordinatorSG: Support GroupMG: Management GroupFP: OCHA Focal Point TD: Target Deadline HC: Humanitarian Coordinator HCT: Humanitarian County Team CAP: Consolidated Appeals ProcessAG: Advisory Group

26. IA RTE Framework & Bottom-Up Approach

28. IA RTE Standard ToR • Focus on participatory approach • Independent consultants team deployed during the initial response phase with arrival no later than two months after the disaster • Assess initial response via desk reviews, field visits, interviews with key stakeholders, especially affected population • & provide real time feedback & learning to the HCT • Consultants serve as facilitators to encourage & assist field personnel to critically examine operations & to find creative solutions to problems • Overarching questions include: • What were the main operational results, and the positive & negative outcomes for all segments of the affected population, during each phase? • Have appropriate common standards been developed within the coordination systems (globally & for each Cluster) & to what degree have these been met?

29. In which ways are IA RTEs different from standard evaluations?

30. IA RTEs are different in the following way: An attempt at system-wide assessment at an early stage Real-Time Learning for the HCT Bottom-Up Approach Specific Trigger Mechanism IA Stakeholders

31. IA RTEs ARE DIFFERENT FROM STANDARD EVALUATIONS DEFINITION: IA RTEs are ‘Inter-Agency (IA)’ in that theRTEs are instigated, managed and used by a variety of international organizations. These are represented at a global level in the IA RTE Support Group and for a specific IA RTE, through the IA RTE Management Group, the Humanitarian Country Team (HCT) and the in-country IA RTE Advisory Group, when established. The term ‘Real-Time (RT)’ refers to theearly implementation stages of a humanitarian emergency operation, to the rapid feedback of IA RTE findings at the field level, and to the possibility of an iterative, multi-phased approach depending on the scale of the disaster. A one-phase approach of an IA RTE would be carried outwithin the two first monthand consists of remote monitoring and the IA RTE mission […]. ‘Evaluation’ (E), as applied in the term IA RTE, describes aparticipatory review process. IA RTEsdifferfrom other humanitarian evaluationsregarding speed, coverage, methods, and outputs.

32. IA RTEs ARE DIFFERENT FROM STANDARD EVALUATIONS PURPOSE: Intended primarily forlearningin emergency operations with field-level‘inter-agency coordination and management at the core, especially in the context of sudden-onset disasters, or protracted crises undergoing a phase of rapid deterioration or escalating violence. Defined as an evaluation carried out at theearly implementation stagesof ahumanitarian operation which almostsimultaneously feeds back its findings for immediate useby the broader humanitarian community at the field level. Seek to identify gaps, access constraints and potential threats to the “humanitarianspace”, assess the relevance, quality and timeliness of the response, unlockinter-stakeholder coordination/ collaboration problems oroperational bottlenecks, propose appropriate strategic re-orientation and provide real time learning to the field on same. Principle aim is toidentify strengths and weaknesses of the response through the assessment of the response to the affected population. Primary end users of IA RTEs are in-country agencies and staff engaged in the humanitarian response, especially those involved in inter-agency coordination and management.

33. Challenges

34. Inter-Agency Challenges • Funding • Coordinating IA Stakeholders • IA-RTE in Complex Emergencies • Responsibility “Sharing”

35. Links Inter-agency Real Time Evaluations http://oneresponse.info/Coordination/IARTE Inter-agency Standing Committee (IASC) http://www.humanitarianinfo.org/iasc/pageloader.aspx OCHA & Inter-agency Evaluation Reports http://ochaonline.un.org/ToolsServices/EvaluationandStudies/ESSReports/tabid/1325/language/en-US/Default.aspx Active Learning Network for Accountability & Performance in Humanitarian Action (ALNAP) http://www.alnap.org/

36. Andreas Schuetz Associate Expert in Humanitarian AffairsPolicy Development and Studies Branch PDSB Evaluation and Guidance Section EGSPalais des Nations, Geneva Tel. +41 22 917 19 29tel. +41 76 271 31 47fax. +41 22 917 00 20E-mail: schuetza@un.org Claude HilfikerSenior Evaluation OfficerPolicy Development and Studies Branch PDSB Evaluation and Guidance Section EGSPalais des Nations, Geneva tel. +41 22 917 11 96fax. +41 22 917 00 20E-mail: hilfiker@un.org Questions?