Measuring the Impact of the RDP

1. Measuring the Impact of the RDP Issues being addressed at an EU level with regards to measuring the impact of the Rural Development programmes B. Schuh www.nrn.ie

2. Content • Assessment of impacts in the RDPs – the basics • Process of assessing RDP impacts • Evaluation architecture/ challenges to overcome • The seven fields of impacts (econ. growth, employment creation, productivity, biodiversity, HNV, water quality, climate change) www.nrn.ie

3. reference for impact supporting and counterproductive trends Assessment of impacts in the RDPs – the basics: • CMEF – the „bible“ • Intervention logic • Additional programme specific indicators • Evaluation questions www.nrn.ie

4. Process of assessing RDP impacts: • Gauging the evidence of change • Identifying the drivers of change • Understanding change and concluding on future interventions www.nrn.ie

5. Evaluation architecture/ challenges to overcome: Factors determinig evaluation architecture: Methodological challenges to overcome: The counterfactual assessment of impacts  quasi-experimental design, non-experimental design; DiD method Taking into account and cross-relating impacts at micro and macro level  e.g. econometric modeling, CGE models, system dynamics modeling Netting out the programme effects by reducing deadweight, leverage, displacement, substitution and multiplier effects data collection and processing  qualitative & quantitative data, FADN & Co Bridging the gap between measuring impact indicators and providing answers on programme impacts  qualitative methods as add-ons – interviews, CS • Dealing with uncertainties • Reducing complexity through a consistent approach • Constraints in utilization of the evaluation results – the evaluation and policy cycle www.nrn.ie

6. The socio-economic impact indicators: • EconomicGrowth: • Calculation via DiD methods, quantification possible, micro-macro link through modeling approaches • Employment creation • Calculation via Propensity Score Matching, Standard regression model, Assessing employment effects at macro level - modelling • Time lag, Missing critical mass – CS, Welfare effects • Labour productivity • Calculation via DiD methods, quantification possible, micro-macro link through modeling approaches • Limitations concerning measuring labour productivity  Competitive Performance, Revealed Comparative Advantage, Growth Competitiveness Indicator, Domestic Resource Cost www.nrn.ie

7. The environmental impact indicators – special issues: • Data availability • Systemic borders • Environmental impacts do evolve also from those measures, which do not deem that such impacts will occur • Conception of the environment within the evaluation (ecosystem functionsvs. ecosystem services) • Difficult to depict the full range of rather vast, complex fields of environmental phenomena like “climate change” or “Biodiversity loss” • “evaluation” vs. assessment – aggregation methods not easily applicable • Cumulative impacts – crossing effects between environmental impacts www.nrn.ie

8. The environmental impact indicators: • Biodiversity • Measured by Farmland Bird Index (FBI) – bottom-up aggregation of micro level observations • Crucial issue of regional/ national specifics, bottom-up assessment, additional information (‘control’ other influences), more than birds • High Nature Value farming/ forestry Measured through: • Land cover characteristics, especially farmland with a high proportion of semi-natural vegetation and in some cases a diversity of land cover types. • Farming practices, especially a low use of inputs (including live­stock density) and specific practices such as shepherding, late hay-cutting, orchard grazing and arable fallowing. • Strong dependence on baseline data  comparison of baseline conditions, main challenge indicator as „work in progress“ www.nrn.ie

9. The environmental impact indicators 2: • Waterquality • Measured as changes in gross nutrient balance (GNB), should be interpreted as a potential risk indicator for water quality only; assessment by bottom up approach – aggregation as methodological challenge (modelling – e.g. RAUMIS) • Many uncertainties remain  different land cover, land use and farming types & atmospheric N fixation and deposition - measuring water quality in agricultural catchment, net nitrogen balance as additional indicators • Climate change • Measured as net greenhouse gas emissions reduction and production of renewable energy • Limitations Additionality/net effects, Displacement of energy & production, Boundary issues, issue of the temporal attribution of longer-term impacts to the policy period of the spending www.nrn.ie