Developing evaluation capacity: Topside Down and Inside Out

1. Developing evaluation capacity: Topside Down and Inside Out Karen Russon, President The Evaluation Capacity Development Group

2. Developing Evaluation Capacity:Topside Down An organization’s evaluation capacity is strengthened through the purposeful systemicdevelopment of an evaluation system… …from the topside downward.

3. Level of ECD Not monolithic activity • National • Organizational • Individual

4. Methods to Promote ECD • Provide information • Create networks • Training

5. Product & Service Mix Matrix

6. To Rent or to Own Should an organization hire an external evaluator (renting) or develop internal evaluation capacity (ownership)? • Moving to Geneva as a metaphor

7. Developing an organization’s evaluation capacity… Begins with a purpose and a shared vision for evaluation. • Accountability • Programming • Promoting Organizational Learning

8. Create a Shared Vision Create a vision for an evaluation system that will enable the organization to meet the desired purpose for evaluation. • Doodling excercise

9. Organizational Design Elements The vision is expressed in terms of organizational design: • Division of Labor – Who will do the evaluation? Specialized staff vs train all staff in basics • Authority – Relationships with co-workers Decision-making authority of evaluation staff vs programming staff • Departmentalization – Where does evaluation reside? Separate evaluation unit vs integrated into programming • Span of Control – Reporting relationships

10. Organizational Culture The vision is also expressed in terms of organizational culture (OC). OC is like the collective personality of the organization: • Observable Artifacts – reports, records, org symbols, stories and myths. • Values – Beliefs about whether evaluation is good or bad. It is a judgment of its worth. • Assumptions – Beliefs about evaluation that we think to be true without questioning.

11. Policies Evaluation policies are the means by which an organization realizes its organizational design (division of labor, authority, departmentalization and span of control). Policies influence the organizational culture (rewards, incentives, promotions, recruitment and staff development).

12. Budgets Policies are operationalized through the budget. • Evaluation budgets range 5-10% • Keep the process simple and budget realistic • Most significant cost is staff time • Data collection and analysis = ½ eval budget; Evaluation design and reporting = other half • Reduce expenses through collaborative efforts with other organizations doing similar activities or joint projects

13. Processes The evaluation system that an organization puts in place will, in large measure, determine the evaluation processes that can be carried out. • Re-examine doodles

14. Benchmarking • Organizations identify and learn from the experiences (best practices) of others either inside their own organization or from other organizations. • A key to effective benchmarking is measuring only what is relevant to the organization’s evaluation vision. • By incorporating benchmarking into the decision-making process, continuous improvement and innovation is promoted.

15. Evaluation Training • Identify the needs: What skills and abilities should be in place? What information should be known by whom? At what point in time should this information be known? • What is the current situation? • The gap between the identified needs and current realities will be filled by evaluation training.

16. Evaluation Standards A helpful resource when developing an organization’s internal evaluation capacity is the United Kingdom Evaluation Society’s Guidelines for Self-Evaluation. http://www.evaluation.org.uk/Pub_library/ Good_Practice.htm#self

17. For more information Evaluation Capacity Development Group (ECDG) and the ECDG Toolkit, see our website: www.ecdg.net Or contact Karen Russon: krusson@gmail.com