1 / 18

Innovative Approaches and New Tools for Evaluation

Innovative Approaches and New Tools for Evaluation. Hans-Martin Boehmer Fall Meetings Evaluation Cooperation Group October 25, 2013 Washington, DC. So much information!! H ow will we make sense of it?. How can we really know what’s happening in the field?.

skyler
Download Presentation

Innovative Approaches and New Tools for Evaluation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Innovative ApproachesandNew Tools for Evaluation Hans-Martin Boehmer Fall Meetings Evaluation Cooperation Group October 25, 2013 Washington, DC

  2. So much information!! How will we make sense of it? How can we really know what’s happening in the field? How can we make our evaluations transparent and participatory? How can we give stakeholders access to our data? How can we disseminate global lessons?

  3. How can we give stakeholders access to our data?

  4. Interesting, Insightful Data is in Demand

  5. IEG data is easily graphed with Tableau

  6. How can we disseminate global lessons?

  7. All staff can access lessons online Evaluation has produced thousands of lessons since 1996 E-LRN 2.0 will: Give WBG staff access lessons from investment, advisory and MIGA operations Stimulate new insights and engage thought leaders Recognize subject matter experts by appointing Knowledge Gurus Encourage discussions around topics and themes Keep staff engaged and informed through alerts and widgets

  8. How can we really know what’s happening in the field? How can make our evaluations transparent and participatory?

  9. IEG uses new technology and online media

  10. IEG collects voices from the Field • Comment from a Facebook user on access to health in Afghanistan: “ unfortunately most of these works [health improvement projects] are limited and they remain only on a piece of paper instead of being implemented. A good example is our district, where nothing has been done yet …” Comment from a user from Ahmad Abad district, Paktia province. [Comment translated from Pashto] Comment from LinkedIn user on SME Definition: “ Within APEC, there is no region- wide accepted definition of SMEs. Definitions vary across members and are based on several different criteria, including number of employees and maximum levels of capital, assets, or sales, which may again vary according to industry/ sector. APEC economies adopt either single or multiple criteria to define SMEs.” – Comment from a staff of the Asia Pacific Economic Cooperation organization

  11. Emerging big, fast, targeted data sources – not yet exploited by evaluation • Twitter feed analysis – Jakarta tweets more than Tokyo! • Real-time environmental data – e.g. monthly global deforestation by province • Calibrate real-time poverty maps from patterns of phone air-time purchases • Conduct targeted surveys by mobile phone • Better use existing library of high quality household surveys • IEG will experiment, on a demand driven basis, with use of these kinds of data

  12. Examples of IEG’s use of outreach • To help in doing IEG’s evaluations: • Crowdsource: public comment on projects slated for ICR reviews, PPARs, XPSRs (possible collaboration with Ushahidi) • For capacity building and popularizing IEG’s missions • Working with CLEAR and with universities, offer training in (part of) ICR review methodology

  13. So much information!! How will we make sense of it?

  14. IEG is striving to achieve improved, more cost effective portfolio analysis • IEG spends significant resources on filtering documents and coding their contents– e.g. identifying projects that deal with early childhood development or traffic safety • IEG will experiment with text analytic software that can help to rapidly sort and classify documents – used by lawyers, journalists • Possibility of outsourcing simple, tedious analytic tasks via internet (e.g., US State Dept used internet volunteers to rapidly geocode project locations)

  15. IEG is focusing on outputs (standardized products & speed) Linking Databases: • Velocity links more than 20 databases including: Bankscope, MIX Market, MFA, WDI, GDF, IEG Ratings, Fitch (Rating History) and Institutional Investor among others. Standardized Products to Speed our work: • Graphs – tables ready to cut and paste. • Comparative financial information entity vs. local and international peers. Easier comparisons. • All the country information in one place: CAS, EIU, Fitch, Central banks. • Case studies and best practices.

  16. IEG is pursuing ways to improve data handling and visualization • IEG is organizing and documenting ratings and project data • Integrate with external data – e.g. national statistics • Shift from Excel to Tableau for easier analysis and more powerful visualization

  17. Innovative ApproachesandNew Tools for Evaluation Hans-Martin Boehmer Fall Meetings Evaluation Cooperation Group October 25, 2013 Washington, DC

More Related