Criterion 1: intellectual merit

1. Criterion 1: intellectual merit • How important is the proposed activity to advancing knowledge and understanding within its own field or across different fields? • How well qualified is the proposer (individual or team) to conduct the project? (If appropriate, the reviewer will comment on the quality of prior work.) • To what extent does the proposed activity suggest and explore creative and original concepts? • How well conceived and organized is the proposed activity? • Is there sufficient access to resources?

2. Criterion 2: broader impacts • How well does the activity advance discovery and understanding while promoting teaching, training, and learning? • How well does the proposed activity broaden the participation of underrepresented groups (e.g., gender, ethnicity, disability, geographic, etc.)? • To what extent will it enhance the infrastructure for research and education, such as facilities, instrumentation, networks, and partnerships? • Will the results be disseminated broadly to enhance scientific and technological understanding? • What may be the benefits of the proposed activity to society?

3. The trouble with Criterion 2 • Lack of guidance about what is expected with regard to Criterion 2 inside and outside NSF • Criterion 2 elements are ambiguous • Interpreted locally by individual program officers

4. ..clear to you what NSF expects…? • “Not fully. I usually consider this criterion to relate to training of students and representation of under-represented groups….” • “Mostly no. Some questions … seem like an exercise in political correctness and nothing else.” • “No. The entire criterion is nebulous and open to a remarkable range of interpretations and responses.” • “The questions are clear enough. What is not clear is how important the answers are to the funding decision.” • “Not really … It would be very helpful to have `examples' or specific input from the program director.”

5. …Criterion 2 an opportunity or nuisance? • “… almost anyone who addresses the impact of their proposed research project on undergraduates will be engaged in meaningless apple pie-ism.” • “[I submit] a scientific proposal, not a proposal for social engineering.” • “…detracts from objectivity in the process and opens the door for progressive politicization of the entire process.” • “Those who do try to involve education/outreach in their research are not rewarded, simply because education takes a lot of time with very little immediate reward.”

6. What approach do you take to satisfying Criterion 2… • “… clearly delineated section of every proposal that describes …audiences, …new materials…, expected outcomes….” • “As a matter of principle, I do not explicitly respond…” • “To be honest, I don't. … I find such promises irritating….” • “My approach is to salute and answer them as satisfactorily as possible. I assume that great responses will not enhance the proposal review a lot, but poor responses could cause a lot of damage.” • “A brief statement about the importance of both teaching and research…. But I keep it very brief, and focus almost entirely on the science of the proposal.”

7. Where Criterion 2 Comes From: A Speculation • Demonstration that public money is well-spent • Counter image of basic science as entitlement • Invitation to give back

8. “The specialization of science is an inevitable accompaniment of progress; yet it is full of dangers and it is cruelly wasteful, since so much that is beautiful and enlightening is cut off from most of the world. Thus it is the proper role of the scientist that he not merely find the truth and communicate it to his fellows, but that he teach, that he try to bring the most honest and most intelligible account of knowledge to all who will try to learn.” --Robert Oppenheimer, 1954

9. “I'm most disturbed by the attitude of some of my colleagues that their conduct of science is an entitlement….No one is saying that every researcher should take the time to do a unit in a 3rd grade class... But I don't think that some type of outreach, in some direction, is too much to ask to add value to the substantial investments already made in research projects.” ...a PI

11. Advance Discovery and Understanding while Promoting Teaching, Training, and Learning Examples of Activities: Integrate research activities into the teaching of science, math, and engineering at all educational levels (e.g. K-12, undergraduate science majors, non-science majors, and graduate students. Participate in the recruitment, training, and/or professional development of K-12 science and math teachers.

12. Develop research-based educational materials or contribute to databases useful in teaching (e.g., K-16 digital library). Partner with researchers and educators to develop effective means of incorporating research into learning and education. Develop, adopt, or disseminate effective models and pedagogical approaches to science, mathematics, and engineering teaching.

13. Huh?

14. fear

15. … mechanisms…better educate the community… • “The website at OPP is a start. …Clear guidelines to PIs and reviewers in RFPs and other communications should be developed and broadly distributed. PDs should be held accountable….” • “…explain with specific examples how these questions would impact the funding or rejection of proposals.” • “… letter from the program director … giving several actual or hypothetical examples….” • “…list specific examples of ways in which Criterion 2 have been met on the NSF Proposal and Review web sites.”

16. Desired PI Attitude Hey, I could do THAT. Shoot, I could do better than THAT. Putting out a formidable Broader Impacts product is in my personal best interest. Putting out a formidable Broader Impacts product is my civic duty. Hey, this could be FUN.

17. How to help... 1. Web resource containing “exemplars” • best examples of achieving broader impact • description of what it is • how I did it • how I overcame my trepidation

18. How to help... 2. Special collection of Web sites • plain-English Web-based review article highlighting the PI’s research and placing it in context • links to relevant popular articles, movies, whatever • explanation of why the research is important • a collection of “living” resources • need for editorial and Web authoring assistance

19. Thematic Collections • organized by: • Exploration Pathways • Compelling Research Questions • Key Topics • Activities

20. Give a man a fish and you feed him for a day. Teach a man to fish and you feed him for a lifetime. ...Chinese proverb

21. How to help... 3. Workshop series 4. Special sessions 5. What about the big research organizations?

22. What we are about... Create a comprehensive program that will help the geoscience research community bring benefits of its research output to a broader community. More broadly... Bring about a cultural change in the geoscience research community.

23. Achieve cultural change via a collaborative network: National Support Network for Broader Impacts primary nodes: • research organizations • professional societies • government agencies secondary nodes • cadre of geoscience researchers Long term goal an interdisciplinary network?

24. “The value system of the university is research, because research provides money....any change will also require a change in the research culture.” ...a PI