Defining Organizational Effectiveness

1. Defining Organizational Effectiveness George Mitchell Transnational NGO Initiative, Moynihan Institute of Global Affairs, Syracuse University April 6, 2010

2. Organizational Effectiveness: Overview • Lack of simple, accessible, relevant and credible information about NGO effectiveness • Stakeholders, such as donors, forced to rely on incomplete, poor quality information • How can NGOs help provide more meaningful information? • Outline • Part I: definition • Part II: disclosure

3. Why Definition Matters • NGOs want to be effective and to communicate their effectiveness to stakeholders, such as donors. • Donors want their contributions to support effective NGOs. • But if stakeholders define organizational effectiveness differently they will employ different criteria and reach different conclusions • The question of organizational evaluation (and therefore reputation, etc.) depends upon the question of definition

4. Evidence from the Transnational NGO Interview Project Defining NGO Effectiveness

5. The Transnational NGO Initiative • Face-to-face, in-depth interviews with 152 leaders (mostly presidents and CEOs) of international NGOs rated by Charity Navigator • Topics included governance, goals, strategies, obstacles, transnationalism, effectiveness, accountability, communications, partnerships and collaborations, leadership, etc. • About 209 hours of digital recording transcribed and coded (quotations and statistics)

6. TNGO Data on Organizational Effectiveness • Interviewer: “Let me ask you about the concept of effectiveness, which is something we all have trouble defining. How does your organization define effectiveness?” • The (non-mutually exclusive) categories for leaders’ open-ended responses: • Resources • Flexibility • Innovation • Expertise • Contacts • Staff/peer competency • Stakeholder satisfaction • Goal achievement • Evaluation • Other (omitted)

7. Organizational Effectiveness: Overall Results

8. Organizational Effectiveness

9. Two Main Definitions of Effectiveness

10. Two Definitions of Effectiveness Outcome accountability Overhead Minimization “to be effective in [DELETED] is to deliver services and assistance to the people of [DELETED] at low cost” “The uh, amount of money that’s actually getting to the, to the field dedicated to the programs…and secondarily the actual impact of projects that you can quantify.”  “…we can’t determine the outcomes, so we measure products. We, we measure outputs. What is sent over you know, what its purpose is, where it goes. We don’t necessarily know how many people it will effect…you can measure success primarily by your outputs.” “our best ace card is our efficiency in terms of how we, we have a low overhead” • “Well I mean ah, for us to be effective is to achieve the programmatic or strategic goals that we’ve identified.” • “We define it as whether or not we are sort of getting the tasks achieved that we set for ourselves.” • “We set important goals and, and we achieve them.” • “if we’ve done the work that we’ve said we would do, that’s...that should be one level of effectiveness.” • “It’s the commitment you make to doing what you said you were going to do.” • “it’s when you are doing what you’re saying you’re doing, that you’re serving your mission...and that you’re able to show that you’re serving your mission…” Goal attainment, promise-keeping, program evaluation Functional expense ratios, accounting for outputs

11. Effectiveness as Overhead Minimization • Why? • Reaction to external pressures (internalization, marketing)? • Underlying cultural belief that low overhead corresponds with organizational worthiness? Evidence? (Urban/Indiana study) • Who? • These leaders are three times more likely to be from larger NGOs (professionals, economies of scale). • These leaders are ten times more likely to be from “amelioration” NGOs (accounting for outputs).

12. Two understandings, two possible systems Assessing Organizational Effectiveness

13. Communication Breakdown? But we accomplished everything we promised… Your organization is ineffective! Your overhead is too high!

14. The Disclosure Problem • Stakeholders make judgments based on the credible information available to them • In lieu of meaningful “effectiveness data,” ratings based on financial data and checklists have become increasingly popular. • Many users implicitly assume such ratings have something to do with effectiveness. • Many donors are forced to rely upon very little information… (anecdote)

15. Two Definitions, Two Possible Disclosure Systems Outcome accountability Overhead minimization Definition: maintenance of normal functional expense ratios, etc. Disclosure: financial information, much of it relying on discretionary classification • Definition: demonstrable goal attainment or promise fulfillment • Disclosure: states ex ante, goals or promises, states ex post How can NGOs disclose more helpful information about organizational effectiveness to stakeholders?

16. The Wrong Format: Statement of Program Service Accomplishments

17. A Disclosure Format for Outcome Accountability

18. Evaluation Challenges • Although most NGO leaders associate organizational effectiveness with goal achievement and evaluation, according to Charity Navigator less than 10 percent of NGOs implement meaningful evaluation systems… • At the end of the day, are NGOs held accountable for outcome achievement? • The basic principles of outcome accountability • Articulate specific goals and identify appropriate indicators • Evaluate progress as rigorously as feasible • Credibly and systematically disclose results

19. Conclusions • The TNGO Initiative website: http://www.maxwell.syr.edu/moynihan_tngo.aspx • Subscribe to the blog. • Email: gemitche@maxwell.syr.edu