Navigating Ethical Challenges The importance of professional mentorships in development evaluation

1. Navigating Ethical ChallengesThe importance of professional mentorships in development evaluation Sarah Mason Independent Consultant

2. Introduction • Development evaluation is particularly vulnerable to ethical challenges. • Navigating ethical dilemmas is daunting for young evaluators given limited training in ethics. • Presents a case study of one young evaluator’s experience in responding to ethical dilemmas. • Highlights the utility of professional mentorships.

3. Development Evaluation Vulnerability arises from three factors: • Underfunding • Evaluations occur over 2 to 3 weeks at the end of a project; • Limits the base for research population; • Complicates data collection on comparison groups. • Data constraints • Absence of solid program records.

4. Development Evaluation 3. Systematic biases that discourage robust evaluation • Absence of an accountability feedback loop; • Incentives for positively biased evaluations; • Tendency to combine Monitoring and Evaluation and Communications / PR roles.

5. Limited Opportunities • Few evaluators have training in evaluation ethics (Berends 2007); • The training we receive emphasises issues of technique and methodology without consideration for the political environment (Chelimsky 2008, p.400); • Ethics standards can increase anxiety (Mabry 1999); • Short term contracts limit ongoing professional development opportunities.

6. Teaching Ethics • Beginner evaluators move through three developmental stages with respect to ethics: • Naïveté and rigidity; • Disequilibrium; • Assimilation. • Students progress through these stages when they receive “supportive mentoring,” encounter challenges and are encouraged to reflect.

7. Background • 2010, one young evaluator won a contract to conduct a summative evaluation in East Timor. • First independently conducted evaluation. • Budget - $3,000. • Found that the project had met three of its four indicators, but these were pre-disposed to positive bias.

8. The Ethical Dilemmas • Two primary ethical dilemmas: • Lack of funds and time constraints led to unrepresentative sample; • The project manager instructed the evaluator to re-write the findings to highlight positive results. • Concern that the evaluation would not assist them in gaining future funding.

9. Responses • First dilemma: acknowledged limitations in the final report. • Second dilemma: sought advice of a senior evaluator. • “I knew how I wanted to respond but did not know if it was the right thing to do.”

10. What can we take from this? Three key lessons: • Access to a more experience evaluator provides opportunities for consolidation; • Confidence is key; • Uncertainty discouraged an initial request for help.

11. Conclusion • Professional mentoring represents a training methodology that responds to all three. • Premise: young evaluator is meant to ask for guidance; senior evaluator is willing to offer support. • Advice provided in highly contextualised ‘real world’ environments. • Cultivate greater practical understanding of evaluation ethics and contribute to higher quality evaluations.