An Assessment Framework for Evaluating Effectiveness of Stream Restoration Projects

1. An Assessment Framework for Evaluating Effectiveness of Stream Restoration Projects P. Srivastava†, J.N. McNair‡, D.D. Hart‡, R.J. Horwitz‡, and J.C. Carr‡ †Biosystems Engineering Department Auburn University and ‡Patrick Center for Environmental Research Academy of Natural Sciences of Philadelphia

2. Lots of effort and money focused on stream restoration and protection • Degradation of streams has resulted in thousands of stream restoration projects to • improve in-stream and riparian habitat • reduce stream bank erosion • minimize the impact of non-point source pollution, and • enhance biodiversity • Billions of dollars are being spent in the United States alone • People are turning from hard engineering solutions to ecologically-based restoration activities

3. Stream Restoration Projects Bank Stabilization Regrading/matting Rootwads Live stakes Brush deflectors/mattresses Gully fills Channel Management Livestock stream crossings Natural stream channel design Rock vanes Rock weirs Log deflectors Habitat Management Riparian fencing Riparian planting Exotic species removal Boulders for fish habitat Riffle creation

4. Idealized Adaptive Management Model Watershed Assessment Watershed Restoration Goals Funding Agency Lead Entities Project Identification Prioritization Funding Implementation Funding Strategy Restoration and Protection Strategy Monitoring Implementation Effectiveness Validation Adaptive Management Adapted from Monitoring and Evaluation Strategy, Washington Salmon Recovery Funding Board, 2003

5. Problem with the idealized adaptive management model • Some agencies (e.g., Washington Salmon Recovery Funding Board) are developing formal monitoring and evaluation strategy • Evaluation strategy (i.e., the adaptive management model) good if a few agencies implement and monitor projects • In many regions, restoration projects are often implemented by local watershed organizations with limited resources and limited technical expertise • Projects are often local in scale with little prioritization or coordination of restoration efforts - limited project evaluation

6. What constitutes a successful stream restoration project? Five Criteria (developed by task force funded by the National Science Foundation called National River Restoration Science Synthesis Project) • Project design should be based on a specified guiding image (i.e., a reference site) • The river’s ecological condition must be measurably improved • River system must be more self-sustaining and resilient to external perturbations • Project construction should not inflict lasting harm on the ecosystem • Both pre and post assessment must be completed and data made available

7. Evaluating stream restoration projects • What type of monitoring should we conduct? • Implementation • Effectiveness, and • Validation • Which ecosystem attributes (also called response variables or indicators) should we monitor? • How often should we sample? • Which reference (control) site(s) should we use?

8. Evaluating stream restoration projects Types of monitoring • Implementation • Used to determine whether the project was implemented in compliance with the project design, and whether the design was adequate for addressing the goals of the project • Effectiveness • Used to determine how well the restoration objectives are being met • Validation • Determines if the key assumptions made in the plan are valid • Focus on implementation and effectiveness monitoring

9. Evaluating stream restoration projectsRiparian Planting with Temporary Stream Bank Stabilization (an example) • Implementation Monitoring • Planting done according to plan • Number of plants survived • Coconut matting still intact • Effectiveness Monitoring • Increased shading of stream - reduced stream temperature (direct effect) • Increased bank stability - reduced sediment load to the stream (direct effect) • Improved habitat and water quality (direct effect) • Improved in-stream biota - periphyton, macroinvertebrate, fish (indirect effect)

10. Framework for evaluating restoration projects • Required to collect relevant project data using consistent methods • Should be scientifically valid and practical • Should be based on our conceptual understanding of how natural stream systems work • Allows us to identify response variables that are most likely to be directly or indirectly affected by particular stream restoration • Which variables will respond in short-term vs. which variables will respond in the long-term • Should allow restoration practitioner to see measurable improvement in ecological condition

11. Framework for evaluating restoration projectsEffectiveness Monitoring • Expected project outcomes should be stated clearly • Sensitive response parameters linked to expected outcomes should be identified • Parameters respond in short term vs. long term • Easy to measure • Sensitive to stresses • Demonstrate predictive responses to stresses (or restoration) • Should be integrative • Ecological indicators are major area of research • Monitoring and statistical design should be clearly laid out

12. An ExampleRiparian planting with temporary stream bank stabilization

13. An ExampleRiparian planting with temporary stream bank stabilization

14. An ExampleRiparian planting with temporary stream bank stabilization

15. Conceptual BasisRiparian planting with temporary stream bank stabilization Temporary Stream Bank Stabilization Improves Stream Bank Stability Decreases Stream Bank Erosion Rates Decreases Embeddedness

16. Conceptual BasisRiparian planting with temporary stream bank stabilization Short-term (1-5 Years) Long-term ( >5 Years) Riparian Planting Riparian Planting Improves Native Vegetation Improves Shading Decreases NPS Loading of Sediment, N, and P Increases Dissolved and Particulate OM, and LWD Improves Stream Bank Stability Decreases Maximum Stream Temperature Improves abundance and diversity of Fish and macroinvertebrate Decreases algal biomass Decreases Embeddedness Improves Diversity Benthic Macro

17. I. Project OutcomesRiparian planting with temporary stream bank stabilization • Short-term (1-5 years) outcomes • Reduce sediment loading to stream from stream bank erosion • Reduce non-point source sediment loading to stream from watersheds (if the project is implemented at a large spatial scale as compared to the watershed size) • Create native, riparian habitat • Improve in-stream habitat for benthic macroinvertebrates • Improve diversity of benthic macroinvertebrate community • Long-term ( >5 years) outcomes • Increase shading • Reduce non-point source N and P loading to stream • Increase inputs of dissolved and particulate organic matter and LWD • Improve abundance and diversity of fish community • Decrease algal biomass

18. II. Identification of Response ParametersRiparian planting with temporary stream bank stabilization Temporary Stream Bank Stabilization Measurable Improves Stream Bank Stability Measurable Decreases Stream Bank Erosion Rates Measurable Decreases Embeddedness

19. II. Identification of Response ParametersRiparian planting with temporary stream bank stabilization Short-term (1-5 Years) Long-term ( >5 Years) Riparian Planting Riparian Planting Measurable Not easy to measure Improves Native Vegetation Not easy to measure Improves Shading Measurable Decreases NPS Loading of Sediment, N, and P Increases Dissolved and Particulate OM, and LWD Improves Stream Bank Stability Measurable Not easy to measure Decreases Maximum Stream Temperature Improves abundance and diversity of Fish and macroinvertebrate Decrease algal biomass Measurable Measurable Decreases Embeddedness Measurable Improves Diversity Benthic Macro

20. II. Identification of Response VariablesRiparian planting with temporary stream bank stabilization • Short-term (1-5 years) • Bank stability (e.g., EPA Visual or Pfankuch Assessment) • Stream bank erosion rate (e.g., Erosion Pins) • Embeddedness (e.g., EPA Visual Assessment) • Canopy density (using densiometer or other better methods) • Vegetation survey (e.g., % areal coverage by canopy, understory, and ground cover; % areal coverage by invasive species) • Benthic macroinvertebrate sampling • Long-term (>5 years) • Fish sampling • Algal Biomass • Large Woody Debris • Response variables difficult to measure • Sediment, N, P, dissolved and particulate OM, temperature, etc. • Photographs from a reference point

21. III. Monitoring and Statistical DesignRiparian planting with temporary stream bank stabilization • Ultimately the goal is to determine if there is a measurable change in physical, chemical, and biological indicators at the restored site as compared to reference (control) site • Several approaches have been proposed for a monitoring design based on Before-After-Control-Impact (BACI) design • BACI design samples the control and impacted sites simultaneously before and after the impact (or restoration) has occurred • The goals is to compare the before and after periods by a t-test for a difference between the means of the before differences and the means of after differences

22. III. Monitoring and Statistical DesignRiparian planting with temporary stream bank stabilization Example of a BACI design Before Calculate di = Ci – Ii for each time i Compare d values between groups (before and after) using a t-test After

23. III. Monitoring and Statistical DesignRiparian planting with temporary stream bank stabilization • Several variations of BACI design have been proposed depending on the situation • Original BACI design (Green, 1979) – SINGLE sample “before” and SINGLE sample “after” at each “impacted” and “control” sites • Problem – the difference might not be related to the human activity but by the confounding effects of natural factors • Extended by Stewart-Oaten et al. (1986) to have several replicated times of sampling (discussed before) • Solved the problems of lack of temporal replication • Doesn’t solve problems caused by the lack of spatial replication

24. III. Monitoring and Statistical DesignRiparian planting with temporary stream bank stabilization • Comparison between a SINGLE “impact” and a SINGLE “control” still confounded by any other cause of different time courses of response in the two places that is not due to the identified human activity • The most robust (and probably most expensive) approach using an asymmetrical design with one impacted and several control locations was proposed by Underwood (1994), Ecol. App. • Can detect impacts that do not affect long-term means, but do alter temporal variance • Asymmetrical designs are also extendable to sample at hierarchical spatial and temporal scales • Great for sustained, press responses (e.g., restoration)

25. III. Monitoring and Statistical DesignRiparian planting with temporary stream bank stabilization • BACI design suggested by Underwood (1994) with one “impacted” and several “control” site is the ideal • How can we reach to this ideal? • State and federal agencies can work together to set up a network of reference sites where they routinely collect data • The sites can then be used as “control” sites for evaluating stream restoration projects • Often not possible for state agencies to do so because of cost and human resources associated with such an endeavor

26. III. Monitoring and Statistical DesignRiparian planting with temporary stream bank stabilization • We propose using a paired “control” upstream of the restored location • Data collected before and after at each of the control and restored location one time will result in a BACI design proposed by Green (1979) • Data collected multiple times before and after restoration will provide sufficient data for Stewart-Oaten et al. (1986) type BACI design • If this procedure is repeated by a number of restoration practitioners it will provide data for asymmetrical design (multiple control and single impact) proposed by Underwood (1994)

27. III. Monitoring and Statistical DesignRiparian planting with temporary stream bank stabilization • Several levels of assessments can be performed using this approach and confidence in inference increases with time and number of projects implemented in the area • The burden of data collection is distributed, however, everybody benefits from each others data • Lack of “before” data can be addressed by using data from other sites • Such an approach requires that restoration practitioners use standard protocols, metadata, and procedures for collecting data • Restoration practitioner make a commitment to long-term monitoring

28. Summary and Conclusions • Funding agencies are desperately looking for ways to evaluate stream restoration projects • A framework similar to the one proposed is needed to evaluate outcomes of stream restoration projects • Clear objectives, sensitive response parameters, and good monitoring and statistical design are essential for project effectiveness evaluation • These three things (objectives, response variables, and monitoring and statistical design) should primarily be based on (among other things) our conceptual understanding of how watershed systems work

29. Summary and Conclusions • Such a detailed and clear assessment framework should be explained to restoration practitioners • Standard protocols, metadata, and procedures should be provided to restoration practitioners • Monitoring based on selected response parameters should be “phased” so that future monitoring can be improved based on the results from initial monitoring

30. Acknowledgement Pennsylvania Department of Environmental Protection William Penn Foundation