Robert Noyce Teacher Scholarship Program: Strengths and Weaknesses of Submitted Proposals

1. Robert Noyce Teacher Scholarship Program: Strengths and Weaknesses of Submitted Proposals • Nicole Bennett, Ph.D. Chemistry • Gregory Goins, Ph.D. Biology • Division of Undergraduate Education • National Science Foundation • AAAS Noyce Workshop • Washington, DC • January 9, 2014

2. NSF Review Criteria • NSF Merit Review Criteria • Intellectual Merit: encompasses the potential to advance knowledge • Broader Impacts: encompasses the potential to benefit society and contribute to the achievement of specific, desired societal outcomes • Additional Noyce Program-specific review criteria, dependent on proposal type

3. Review Criteria: Phase I Scholarship Proposals • Capacity and ability of institution to effectively conduct the program • Number and quality of students that will be served by the program • Justification for number of students and amount of stipend & scholarship support • Quality and feasibility of recruitment & marketing strategies Strong: Provides data to justify need and realistic expectations; indicates number of participants Weak: Projections not supported by data

4. Review Criteria: Phase I Scholarship Proposals • Ability of the program to recruit STEM majors who would not otherwise pursue a teaching career Strong: Indicates they will recruit beyond those who are already in the program Weak: Not expanding beyond current pool

5. Review Criteria: Phase I Scholarship Proposals • Quality of the pre-service educational program Strong: • Provides detailsabout program • Provides evidence that graduates are successful • Research based Weak: Little information provided

6. Review Criteria: Phase I Scholarship Proposals • Extent to which STEM & Education faculty are collaborating in developing and implementing the program Strong: Good representation of STEM and education faculty; defined roles in management plan; shared responsibility. Clearly identified as Co-PIs, senior personnel, or other with biosketch provided. Weak: No evidence of collaboration (“in name only”)

7. Review Criteria: Phase I Scholarship Proposals • Quality of the infrastructure to support pre-service students and new teachers Strong: A clear plan for supporting students and new teachers to ensure success; strong partnership with school district Weak: No support beyond the financial support • Extent to which the proposed strategies reflect effective practices based on research Strong: based on literature and research findings Weak: no references or not clear how the project is based on research

8. Review Criteria: Phase I Scholarship Proposals • Degree to which the proposed programming will enable scholarship or stipend recipients to become successful mathematics and/or science teachers • Strong: Program designed to address specific needs of Noyce Scholars, in particular the high-need school setting • Weak: Program does not appear to be designed to support needs of Noyce Scholars

9. Review Criteria: Phase I Scholarship Proposals • Feasibility and completeness of an evaluation plan that will measure the effectiveness of the proposed strategies Strong: an independent evaluator; clear objectives and measures; describes data collection and analysis aligned with evaluation questions and project goals Weak: No objective evaluator; evaluation not aligned with project objectives

10. Review Criteria: Phase I Scholarship Proposals • Institutional support for the program and the extent to which the institution is committed to making the program a central organizational focus Strong: Evidence of support from departments and administrators; likely to be sustained; integrated with other STEM initiatives Weak: Lack of supporting letters from Administrators; little involvement beyond the PI

11. Summary of Common Weaknesses • Proposal does not follow guidelines for Noyce Program • Students must complete STEM major (not change to Science Education or Math Education major) • Little information about teacher preparation program • Unrealistic projections • Recruitment and selection strategies not well described • Lack of support for new teachers • Lack of involvement of STEM faculty (or Education faculty) • Lacks plans for monitoring compliance with teaching requirement • Weak evaluation or lack of objective evaluator • Does not address Prior Results; Lessons Learned • Lacks details

12. Review Criteria: TF/MTF Proposals • Capacity and ability of institution to effectively conduct the program • Number and quality of Fellows that will be served by the program • Justification for number of Fellows served and amount of stipend and salary supplements • Quality & feasibility of recruitment and marketing strategies

13. Review Criteria: TF/MTF Proposals • Extent to which the proposed strategies reflect effective practices based on research • Degree to which the proposed programming will enable the participants to become successful mathematics and science teachers or Master Teachers • Extent to which STEM and education faculty are collaborating in developing and implementing a program with curriculum based on the specialized pedagogy needed to enable teachers to effectively teach math and science and to assume leadership roles in their schools.

14. Review Criteria: TF/MTF Proposals • Feasibility & completeness of an objective evaluation plan that will measure the effectiveness of the proposed strategies • Institutional support for the program & the extent to which the institution is committed to making the program a central organizational focus • Evidence of cost sharing commitments • Plans for sustainability beyond the period of NSF funding

15. Review Criteria: TF/MTF Proposals NSF Teaching Fellows only: • Ability of the program to recruit individuals who would not otherwise pursue a career in teaching and to recruit underrepresented groups • Quality of the Master’s degree program leading to teacher certification • Quality of the infrastructure to support pre-service students and new teachers NSF Master Teaching Fellows only: • Quality of the professional development that will be provided

16. Strong TF/MTF Proposals include • Strong partnership with school district • Matching funds identified • Clear description of pre-service program for Teaching Fellows and professional development program for Master Teaching Fellows • Detailed recruitment and selection plans • Clear vision of Master Teacher roles and responsibilities, including involvement in pre-service programs • Attention to content and pedagogy • Detailed evaluation plans

17. Weak TF/MTF Proposals may show • Insufficient detail for pre-service and induction programs for Teaching Fellows and professional development program for Master Teaching Fellows • Vague recruitment plans • Selection plans not according to guidelines • Master Teacher roles and responsibilities not discussed • Matching funds not identified • Role of non-profit organization not clear • School district partnership not strong • Evaluation weak or lacking independence

18. In a Strong Partnership • Individuals from all institutions have clear roles and communication structures • Management plan includes a description of how communication, meetings, roles, division of responsibilities, and reporting will occur • Distribution of resources is appropriate to the scope of the work • All partners contribute to the work and benefit from it • Letters of commitment are provided

19. The Process • Proposals may be submitted to FastLane or grants.gov (Use FastLane for TF/MTF proposals) • All proposals are peer-reviewed according to standard NSF merit review criteria (Intellectual Merit and Broader Impacts) • Notification of results within six months of receipt • Reviewers’ comments may be accessed through FastLane after final decision is made

20. What Makes a Proposal Competitive? • Original ideas • Succinct, focused project plan • Realistic amount of work • Provision of sufficient detail • Cost effectiveness • High impact • Knowledge and experience of PIs • Contributions to the field • Rationale and evidence of potential effectiveness • Likelihood the project will be sustained • Solid evaluation plan

21. Tips for Success • Consult the program solicitation and NSF Proposal and Award Policies and Procedures Guide (NSF 14-1) • Test drive FastLane • Alert your institution’s Sponsored Research Office • Follow limits on page margins and font size • Be aware of other projects and advances in the field • Cite the literature as appropriate • Provide details • Discuss prior results • Include evaluation plan with timelines and benchmarks

22. Tips for Success • Put yourself in the reviewers’ place • Consider reviewers’ comments if resubmitting (keeping in mind that they will be different people) • Have others read the proposal • Spell check; grammar check; attend to details • Meet deadlines • Follow NSF requirements for proposals involving Human Subjects • Call or email NSF Program Officers as needed

23. FastLane will not accept if: • Fail to describe mentoring activities for postdoctoral researchers if any included in proposed budget • Fail to include data management plan • Fail to include complete Project Summary (3 text boxes: Overview, Intellectual Merit and Broader Impacts statements)

24. Return Without Review • Submitted after deadline • Fails to follow formatting(e. g. page limitation, font size, and margin limits) requirements

25. Not ready to submit a proposal this year?Consider serving as a reviewer.Send a letter of interest and a CV to one of the program officers or fill out the online form athttps://www.surveymonkey.com/s/NSF_DUE_Reviewer_Info

26. For more information: www.nsf.gov (go to Division of Undergraduate Education) www.nsfnoyce.org Or contact us: Joan Prival jprival@nsf.gov Nicole Bennett nbennett@nsf.gov Gregory Goins ggoins@nsf.gov