Connecting the Dots between Research Priorities and Stakeholder Needs: An Evaluation of Stakeholder Perceptions of

1. Connecting the Dots between Research Priorities and Stakeholder Needs:An Evaluation of Stakeholder Perceptions of theNSRC Grant Program Curt Grimm, Ph.D. Barbara Wauchope, Ph.D. Charlie French, Ph.D. Carsey Institute, UNH Durham, NH NSRC Leaders Meeting Hubbard Brook Foundation North Woodstock, NH October 11, 2011

2. Purpose of the Evaluation • To assess the work of the NSRC from the perspective of multiple stakeholders • NSRC grantees – Principal Investigators on research grants • Practitioners and others who partner with University grantees • Other practitioners, policy-makers, and citizens who have interest in/use for the research

3. Areas of Evaluation • NSRC grantmaking process • Current research needs and priorities re: Northern Forest • Relevance and usefulness of the current NSRC research focus • Mechanisms grantees use to communicate research results • Communication/outreach strategies that might be more effective

4. Methodology:Data collection Online Surveys: • NSRC grantee Principal Investigators (n=80) • Users or beneficiaries of NSRC-funded research – identified by the grantees in their survey (n=9) • Actual or potential users or beneficiaries of NSRC-funded research – subscribers to the Northern Forest Center Digest subscriber list (n=69)

5. Methodology:Resulting samples – PI Survey (n=80)

6. Methodology:Resulting sample – Stakeholder Surveys combined (n=78)

7. Methodology:Data collection Telephone interviews: • Actual or potential users or beneficiaries of NSRC-funded research – list generated by NSRC theme directors and Carsey staff (n=24)

8. Methodology:Resulting sample – Stakeholder interviews (n=24)

9. Methodology:Data collection & resulting samples Focus groups/informal interviews: • Northern Forest Higher Education Resource Network members (n=12) • University/college presidents, deans, faculty • Cooperative Extension county-based educators (n=4) • Forest and wildlife staff • New Hampshire Local Government Center employees (n=2)

10. Key Findings • Perceptions of NSRC are generally positive • Scientists and non-scientists have somewhat different perspectives • All agree on the need for both basic and applied research • Scientists somewhat more supportive of need for basic research • Non-scientists overwhelmingly say science should address practical problems

11. Key Findings • All four themes are relevant and important but #1 is of strongest interest: the economy • Current dissemination strategy is not effective • E-mail-based and colleague-based outreach strategies, either your own or through existing networks, are likely to have more impact

12. NSRC GrantmakingPI experience with NSRC (n=80) • Number of awards: 0 – 6 • Mean: 1.89 • Median: 2

13. NSRC GrantmakingPIs: How they hear about Request For Proposal

14. NSRC GrantmakingPIs: Themes broad enough? (n=76)

15. NSRC GrantmakingPIs: Effectiveness of grantmaking process (n=76)

16. NSRC GrantmakingPIs: Effectiveness of mission (n=70-76)

17. NSRC GrantmakingPI Perceptions of the program • Positive about program overall, including its efficiency and effectiveness • Promotes useful research relevant to the Northern Forest region • Promotes collaboration – between scientists, across disciplines, with non-scientists • Increases research on the Northern Forest • Supports a broad scope of projects and disciplines • Promotes interdisciplinary research

18. NSRC GrantmakingPI Perceptions of the program • “Without this program there would be very little research that directly deals with issues important to this region.” • “done well to fund a breadth of projects and expertise across disciplines and regions” • “ exceptionally well-positioned to allow researchers to leverage resources from other sources” • “great collaboration and synthesis within themes. Less opportunity to cross themes.”

19. NSRC GrantmakingPI Concerns • Grant focus • Not relevant enough to Northern Forest residents/too relevant – important scientific questions not funded • Not enough social science • Limitations created by small size of grants • On research topics imposed by themes • On scope that’s possible and resulting impact • On fostering grad students • Grantmaking process: • Scientific review process not followed in some cases • Poor communication to rejected applicants • Bias in awards

20. NSRC GrantmakingPI Concerns • “projects in isolation fail to convey full impact – needs a vehicle for nesting and connecting to NSRC’s work in the big picture” • “funding pool is so small that it’s hard to develop a good collaborative, cross-disciplinary proposal” • “not all themes are equally "open" to competition” • “the themes divide up the pie and a proposal that crosscuts themes is disadvantaged” • “short proposal precludes effective review by panels who are quite mixed in discipline, depth, and breadth of knowledge”

21. NSRC GrantmakingStakeholder awareness of NSRC (n=76)

22. NSRC GrantmakingStakeholder understanding of NSRC (n=15) • What is the program focus: • Research (14) • Funding for research (11) • Balance of applied/basic science research (3) • Applied research (2) • Basic science research (1) • Who NSRC serves: • Broad range of stakeholders (8) • Targeted audience (5)

23. NSRC GrantmakingStakeholder collaboration on grants (n=75)

24. Research Needs & Priorities: Perceived benefits of NSRC research

25. Current NSRC Focus: Themes of present research reported by PIs

26. Research Needs & Priorities:Stakeholder interest in NSRC themes

27. Current NSRC Focus:PI reported themes vs. Stakeholder interest 4

28. Research Needs & Priorities :Stakeholders: Topics of interest to be addressed:

29. Other PI-Perceived Research Needs and Priorities: • Understanding local community need and demand for biological data (and how to get them what they want) • Effects of forest habitat supply on featured wildlife species • Forest hydrology • Forest – stream ecosystem connections

30. Other Stakeholder Perceived Research Needs and Priorities: • Effects/impacts of renewable energy development • Climate change impacts on economy, community, ecosystems…; and vice-versa • Tourism’s potential to turn around the current economic downturn • Local food and agricultural systems impact on economy and ecology • Economic and ecological impact of community forests

31. Stakeholders Emphasize Healthy Economy and Communities • “What’s true in the rainforest is true in the Northern Forest – we have to deal with the poverty” • “If we don’t have a healthy community, we won’t have a healthy forest. A healthy forest is a working forest” • “really integrative, holistic collaborative engaged work is a far better way to go. I would really like to see that value really written into the RFP and the grant proposal evaluation process”

32. Relevance and Usefulness of NSRC Foci: PIs: Relevance of NSRC Research by Audience

33. Relevance and Usefulness of NSRC Foci: PIs: Do you use results of other NSRC research

34. Relevance and Usefulness of NSRC Foci: Stakeholder perceptions • “absolutely critical…it’s given us the basis of our economic development program” • “helpful in that it confirms other research that has been conducted “ • “the one that I collaborated with was really valuable to my department, and my staff” • “great information for scientists and researchers” • “able to expand my own work through channels of NSRC project PIs”

35. Relevance and Usefulness of NSRC Foci Stakeholder perceptions (continued) • “haven’t found them to be very relevant; too theoretical and abstract … most of the work is ecologically focused and not relevant for communities and specifically for their economic growth” • “I can’t think of anything that has had a direct impact” • “I don’t feel really connected to the research to be honest … but I can see the potential” • “there doesn't seem to be an emphasis on application of the research in the real world” • “some of it, truthfully, just goes over my head”

36. Current Outreach & Dissemination: PIs: Most important products of their research

37. Current Outreach & Dissemination: PIs: Which resources are they using

38. Current Outreach & Dissemination: Stakeholders: Where they go for information

39. Current Outreach & Dissemination: PIs: Where do they go to find NSRC research

40. Current Outreach & Dissemination: PIs: How effective are resources for communication

41. Current Outreach & Dissemination: Stakeholders: Average likelihood of using source

42. I rely on my colleagues to stay up to date with the latest research findings. That is where I learned about NSRC. - Focus Group Participant, Forestry Practitioner

43. Current Outreach & Dissemination: PIs: Barriers to dissemination of their research

44. Current Outreach & Dissemination: Stakeholders: Barriers to obtaining research

45. Current Outreach & Dissemination: Stakeholders: Other barriers to obtaining research • Information is not always summarized well into one or a few sources (it's all over the place) • Restricted travel to meetings • Cost to obtain peer reviewed articles • “Lack of easy access to information about the grants and the project findings is a challenge, as is lack of time to sort through lots of information”

46. Current Outreach & Dissemination: Stakeholders: Effectiveness of NSRC dissemination • Some potential audiences are not aware of NSRC • ‘Forestry practitioners’ and forest-serving institutions typically hear about NSRC research findings through word-of-mouth. • There is no mechanism to drive people to NSRC’s website. • All 14 of those interviewed who knew of NSRC said dissemination was ineffective

47. Alternative Dissemination Strategies: PI Suggestions • Online access to publications and perhaps even data collected for each project • Issue a NSRC press release / press conference after each project year, with a bit about each project • More face-to-face contact with research users/stakeholders (but that takes time and resources) • develop an outreach document summarizing project results and management implications

48. Alternative Dissemination Strategies: PI Suggestions (continued) • Work harder with local stakeholder groups such as environmental managers • More informal outlets (workshops for professionals, articles in general interest magazines or newspapers) • More informal public presentations and seek co-sponsorship with aligned local organizations • A more interactive and content-rich website

49. Alternative Dissemination StrategiesStakeholder suggestions • NSRC should actively reach out to practitioners and other stakeholders. • Use Northern Forest Center’s News Digest to disseminate key findings and links • E-mail announcements with links to websites • Better use of forest-serving organizations’ websites to share research and grant info

50. Alternative Dissemination StrategiesStakeholder suggestions (continued) • Ensure that info being shared is up-to-date. • Keep summary of findings short and simple and link to a website for more information. • Just getting an email saying, “hey, we updated the project list and here is what is going on across the region” would be helpful… don’t even need to send the project abstracts… just knowing that they have been updated is helpful.”