Developing Evaluation Capacity

1. Developing Evaluation Capacity Jean Langlois, MSc, MBA

2. Outline • Intro • Nature of the work • NGO funding model • Motivating people • Leading questions

3. Introduction • The world according to Jean Langlois… • Challenges in developing evaluation capacity at an ENGO • Nature of the work • NGO funding model • Motivating people

4. Nature of the work - Challenges Advocacy: • Diffuse causality • Causality attribution Direct Conservation: • Measurable vs. Important biases • Durability of outcomes • Defining outcomes and impacts

5. Nature of the work What works: • Invest time in the logframe at the outset • Articulate and track tangible results • Articulate, don’t track, intangibles What’s needed: • Shared comfort with unmeasured intangible benefits

6. Nature of the work - Questions • How do we combat the biases of the measurable vs. the important? • Is it useful to articulate intangible benefits that will not be measured? • For advocacy campaigns, is there a better model than the dotted-line-logframe?

7. NGO funding model - Challenges • Not enough money, ever • www.naturecanada.ca • Reporting to multiple funders • Multiple evaluation frameworks • Varying degrees of buy-in to rigorous evaluation • Propensity to provide “positive” reports

8. NGO funding model What works: • Invest time in logframe at the outset • Consistent deliverables across grant proposals What’s needed: • Shared expectations • Comfort reporting and learning from failure

9. NGO funding model - Questions • How do we create an environment where people are comfortable sharing and learning from “failure”?

10. Motivating people - Challenges “The effectiveness of our program will be measured by the degree to which the outputs and outcomes identified in the logical framework are achieved.” - NOT Martin Luther King Jr

11. Motivating people - Challenges • “Evaluation is a time drain” • “It’s not my job” • “I’d get more work done if I didn’t have to report on everything I do” - Anonymous Colleagues

12. Motivating people What works: • Adapt the language to the audience • Always relate to mission • Tough love: invest in logframe at the outset What’s needed: • Positive experiences with evaluation • Project-to-project application of lessons learned

13. Motivating people - Questions • How can we create short-term wins (i.e. positive experiences with evaluation)? • How can we design evaluation approaches that produce useful results that are easily applicable to multiple future projects?

14. Leading questions How do we combat the biases of the measurable vs. the important? Is it useful to articulate intangible benefits that will not be measured? For advocacy campaigns, is there a better model than the dotted-line-logframe? How do we create an environment where people are comfortable sharing and learning from “failure”? How can we create short-term wins (i.e. positive experiences with evaluation)? How can we design evaluation approaches that produce useful results that are easily applicable to multiple future projects?

15. Merci