Assessing On-the-Ground Progress of Ecosystem Management Projects

1. Assessing On-the-Ground Progress of Ecosystem Management Projects

3. Oak Openings Working Group • Home to 179 rare plants & animals • Collaboration involving TNC, Toledo Metroparks, • Ohio DNR, US FWS, Toledo Zoo, Others. Oak savanna community intermixed with wet prairie Covers more than 130 square miles near Toledo

5. Alternative Forms of Outcome Assessment Ecological & Social Indicators Participant Self-Assessment Expert Judgment Objectivity Subjectivity

6. Steve Yaffee Steve Brechin Jim Diana Julia Wondolleck Don Zak Elizabeth McCance Steve Higgs Sarah Kopplin, Sheila Schueller Research Team

7. Research Question/Tasks #1 • What are the challenges facing outcome-oriented assessment at real-world sites? How do EM projects actually define success? • Methods: Case studies of 5 sites • Chicago Wilderness IL, Fish Creek IN/OH, Oak Openings OH, Chequamegon-Nicolet NF WI, Kankakee River Basin IN

8. Research Question/Tasks #2 • How have evaluation questions, methods and indicators been defined in the many disciplines associated with EM? • Methods: Literature review: • ecological outcomes • social outcomes • process characteristics

9. Research Question/Tasks #3 • Can we imagine ways to enhance the capacity of EM sites to do effective evaluation? • Method: Brainstorming of a conceptual evaluation process; tested through a Charette • place-based • multi-metric (system states, threats, strategies) • tested through Charette

10. Products • Working paper and draft article on challenges • Working papers summarizing literature on measures of ecological and social outcomes and process characteristics • Draft evaluation protocol and “guide” • Successful funding request to Hewlett Foundation ($200,000 to continue work)

11. Challenges to Effective Evaluation

12. Challenges to Effective Evaluation • Objectives are rarely clear and agreed upon • Few absolutely “correct” normative directions • Ecological/biological processes are not well understood • Systems and strategies operate at multiple scales • Collaborative projects move through different stages • Lack of motivation; limited capacity • Standard evaluation issues: controls, causality

13. Challenges Tendency is to … • Use process improvements as proxies • Measure success of strategies/activities • Measure what is easiest to measure • Focus on data collection, not decision-making • Avoid evaluation, or do it poorly

14. Some Responses

15. Responses • View evaluation as a process integrated with and integral to the success of a collaborative process • As a key part of the process, specify/clarify: • System components and interconnections • Objectives • Reference state/Comparison • Consider motivation and capacity of members of project • Make data needs as simple as possible • Ensure that evaluation produces recognizable benefits to project • Build capacity • Link evaluation to re-assessment & decision-making

16. Responses • Use multiple measures of success: • ecological/biological and social outcomes and process characteristics • ultimate outcomes, nearer term outcomes, and measures of strategy success • Target measures of success to specific phases of project lifecycles. • Be strategic in use of indicators. • keystone; integrative; simple • Use a mix of evaluation approaches: • Subjective: participant self-assessments, expert assessments • Objective: measures of threat reduction, state of the environment, strategy success

19. Elements ofEvaluation Process I. Develop a Situation Map II. Develop an Assessment Plan III. Implement an Evaluation Program

20. I. Developing the Situation Map

21. I. Developing the Situation Map

22. OAK OPENINGS PROJECT SITUATION MAP Strategies Activities Contributing Factors Indirect Threats Direct Threats Target Conditions 1A. Secure funds/support to expand reserves Development (residential, commercial, industrial) Zoning policies Pond construction Sump pumps Individual large lots Roads/Utility lines Habitat loss 1. Expand high quality natural areas 1B. Identify/purchase corridors between preserves 1C. Restore nature on other public lands Groundwater lowering Habitat fragmentation 2A. Restore natural processes (fire, floods) Fire suppression Healthy oak savanna Water wells Ditching Loss of native species 2. Restore environmental quality to existing preserves 2B. Eliminate exotic species 2C. Collect native seeds Agriculture Invasion of exotics 2D. Restore native plants 2E. Reintroduce species Logging/Forestry (planting pines) Stronger community ties to environment Invasion of woody plants 3A. Work with local planning boards to promote env. policy Preserves across multiple jurisdictions 3. Create a more uniform land use policy Insecticide spraying 3B. Develop a generic guide to environmental land use policies for counties Citizens not interacting with nature Intentional displacement of native plants w/ exotics 3C. Coordinate policies between townships Minimal interaction among stakeholders Lack of public env. awareness Sensitive urban growth 4. Engage local politicians in conservation/restoration plans 4A. Build support for Greenways/Green Ribbon Plan Loss of large mammals Development pressure from expanding Toledo 4B. Garner political pressure from influential people Stable and diverse economy Absence of shared goals/objectives Lack of management 5. Educate general public about Oak Openings Uncoordinated land use policy between townships 5A. Create community activist group in Oak Openings 5B. Conduct radio/TV ads Lack of coordination among stakeholders Limited sharing of information 5C. Conduct nature walks Limited agreement on strategies 6. Diversify businesses 6A. Invest in new companies Lack of education about Oak Openings Infrastructure constraints (low airport capacity) 6B. Develop model sust. development policy Limited resources education/research 7. Improve job security Created by Elizabeth McCance and Stephen Higgs; EMI; 3/23/02; O Drive/Staff Document Archive/Stephen Higgs/Oak Openings Information/Oak Openings EM Conceptual Map

23. Steps: 1. Define Measurable Components of Targets & Threats Define Reference - Ideal - Stated goal - Trends - Prior condition or reference site 3. Identify Data for Target and Reference II. Developing an Assessment Plan 4. Prioritize: Putting It Together - Prioritize components - Look for common indicators - Consider resources available and costs

24. Steps: 1. Measurable Componentsof Target Reference - Ideal - Stated goal - Trends - Prior condition or reference site 3. Data Examples: - presence of certain rare species;# of acres of A quality savanna - historic acreage of savanna - 1000 acres of A quality savanna - change over time in #s of acres - historic acreage of savanna - target: aerial photos - reference: survey records II. Developing an Assessment Plan 4. Prioritize: Putting It Together

25. III. Implementing an Evaluation Program Steps: • Determine Who, What, How, When. • Find/Collect Necessary Data • Establish baseline • Look at preexisting sources of data • Evaluate the Target Condition against Reference • Link Learning to Decision-making through On-going Collaborative Process