ISE Preliminary Proposal Virtual Panel PANEL ORIENTATION

1. ISE Preliminary Proposal Virtual PanelPANEL ORIENTATION

2. Program Overview • Main Changes in the New Solicitation • Audiences & Deliverables • Reviewing Proposals (Confidentiality & COIs) • Preliminary Proposals • Virtual Panel Process • Preparing Effective Written Reviews • Navigating FastLane • Questions Contents

3. ISE Program Overview

4. ISE supports innovation in anywhere, anytime, lifelong learning through investments in research, development, infrastructure, and capacity-building for STEM learning outside formal school settings. ISE Program Overview

5. Posing of 6 field-wide challenges • Call for stronger infrastructure • Expansion of professional audience to include volunteers and care-givers • Some STEM topics emphasized • Supplementary docs rules have changed. But for prelims, no need to read supp docs (bio sketches, current & pending, letters of support). Main Changes in New Solicitation

6. Softening of the formal/informal boundary • Issues of national importance in support of seamless STEM learning. • Proposals still need a significant component of learning in an informal setting, but not necessarily a “primary” informal audience. Main Changes in New Solicitation

7. Audiences & Deliverables

8. Public Audiences – People of all ages engaged in STEM learning in informal settings • Professional Audiences – Professionals, volunteers, and caregivers who support and mediate the learning of others. Special Interest: Underserved and Nontraditional Audiences Who are ISE Audiences?

9. Permanent and traveling exhibits • Film, radio, and TV • Cyber-enabled learning, emerging technologies, and educational games • Citizen science projects • Youth and community projects • Research and development in the field • Professional development • Consortia and collaborations • Virtual networks What are examples of ISE Deliverables?

10. Capital or operating expenses • Major or office equipment • Vehicles • Undergraduate tuition • Paid advertising • Admission fees • Operating expenses for school field trips / camps / competitions • Projects that focus primarily on health or medicine • Publications and curricula as the primary deliverable What is not funded by ISE?

11. Reviewing Proposals: Confidentiality & Conflicts of Interest

12. Very strict • Share nothing (even with grad student / close colleague) • Ever… • Shred / destroy paper and electronic versions • Confidentiality Form • Attached to your panel notification letter • Complete this as soon as possible • Send to cemorgan@nsf.gov OR fax to (703) 292-9044. Confidentiality

13. Affiliation with an Institution • Employment (w/in last 12 mos.) • Financial or other gain • Relationship with PI • Joint publication (w/in last 48 mos.) • Financial partnership • Close friendship • Family friend or member What are Conflicts of Interest?

14. Verify first with your Program Officer • If you do have a COI: • Go to the FastLane proposal review page • Scroll to the bottom of the page • In the COI box, write the reason • Press “submit review” button Handling Conflicts of Interest

15. ISE Preliminary Proposals

16. To encourage or discourage full proposal submissions based on the likelihood it will be competitive in the full proposal round. Reviewers typically encourage about 40% (but panels vary). • Provide feedback to PI on the strength and weaknesses of the proposal. Purpose of Preliminary Proposals

17. Allowed 6 pages in length • Summary budget only • No supplementary documents Preliminary Proposals

18. Virtual Panel Process

19. Much of the time you work alone, writing and revising your reviews. • Each proposal has three reviewers who draft their own reviews, then talk about them and change them if they want to. Ideally, all proposals will be discussed, but this is up to your program officer. You can discuss your reviews via email or using your personal phones. • You should participate in conference call discussionswith program officers, at specified times (usually twice per day). Let your program officer know if there are any times you will not be available. • Do rough drafts before the panel if you can. ISE Panelist Responsibilities

20. Preparing Effective Written Reviews

21. Program Officer • PI/Co-PI All reviews will be sent anonymously to the PI/Co-PI. Who Reads Your Review?

22. Preparing Reviews for FastLane (See: Reviewer Guide) Reviewers should address the proposal’s strengths & weaknesses in each section. Section 1: Intellectual Merit (~200 words) • Significance of Issue • Audience & Approach (strategic, appropriate) • Project Design, Methods, & Deliverables (quality) • Innovation (sources, degree) • Qualifications (team, partners, and their collaborative process) • Prior NSF Work(nature and quality, if applicable) Overall Structure of a Review

23. Preparing Reviews for FastLane (See: Reviewer Guide) Reviewers should address the proposal’s strengths & weaknesses in each section. Section 2: Broader Impacts (~200 words) • Advancement (research and/or practice) • Evaluation (learning impacts, thorough, realistic, and appropriate to goals) • Dissemination (breadth, plausibility, specificity) • Participation (of underrepresented groups) • Capacity/Infrastructure (builds) • Other Benefits to Society (if applicable) • Post-doctoral Mentoring Plan (if applicable) Overall Structure of a Review

24. Preparing Reviews for FastLane (See: Reviewer Guide) Reviewers should address the proposal’s strengths & weaknesses in each section. Section 3: Summary Statement (~100 words) • Summarize the fundamental strengths and weaknesses of the proposal (not a place to introduce new material). • Include the sentence “I encourage [or I discourage]” submission of a full proposal. “ • Focus on assessing the quality, not describing the proposal. Be explicit and constructive in your comments. Overall Structure of a Review

25. Only a budget summary is submitted. • (Optional): Is the size and distribution of the budget summary aligned with the proposed project activities? Project Budget

26. 1) Research Projects • Is the research question clearly stated?  • Has related research, from any relevant field, been adequately reviewed? • Is the methodology appropriate, including analysis? *Research projects can be funded for up to $1.5 million and five years in duration. Project-Specific Considerations • 2) Pathways Projects • Is it clear that the proposal is for planning activities that will lead to a full scale proposal or project? • Are the planning activities appropriate and are they likely to help answer critical questions before a major project proposal is submitted (Research, Full-Scale Development or Broad Implementation)? • *Pathways projects can be funded for up to $250,000 and two years in duration.

27. 3) Full Scale Development Projects • Is there evidence that sufficient planning has taken place already and a full scale project is ready for implementation? • Does the project team have prior experience in managing a full scale project with advisors, partners, collaborators, contractors and a large budget? If there are multiple deliverables (e.g. media, exhibits, programs), is there a clear rationale for how these will be integrated or used to achieve the desired impacts? *Full Scale Development projects can be funded for up to $3 million and five years in duration. Project-Specific Considerations

28. 4) Broad Implementation Projects • Is there substantial evidence that the prior work was successful with the targeted audience (including findings from the summative evaluation)? • Does the proposal make clear exactly how they are expanding the reach of the prior project (e.g. in terms of geography, age, socio-economic status, cultural/linguist group or gender)? • If new products are being developed to help with the expanded reach, do they complement and become integrated into the prior work? *Broad Implementation projects can be funded for up to $3 million and five years in duration. Project-Specific Considerations

29. Reviews and recommendations to “encourage or discourage” can be adjusted during the panel. However… • All final reviews and proposal recommendations are due in FastLane by the end of the last day of the panel. Reviewer Deadlines

30. Read all the proposals • See how each proposal responds to NSF/ISE criteria • Prepare reviews in Word; cut and paste into FastLane • Avoid bullets, apostrophes, and quotations (or turn off smart quotes) • Use “spell check” prior to entering review in FastLane • Check review in FastLane to confirm it was entered for the correct proposal and has no weird symbols introduced during pasting. • To modify a review, follow FastLane instructions for resubmitting it Preparing Your Review

31. When you think you might have a conflict of interest after reading a proposal. • When you are not clear about some of the criteria and how to apply them. • When you aren’t sure if your comments are appropriate. • When you have questions and need guidance. Call Program Officer for Advice…

32. Questions

33. Navigating FastLane

34. Reviewer handbook • Logging in • Changing Your Password • Updating Personal Information • Reimbursement/Banking information • Accessing Proposals & Requesting Copies Getting Started in FastLane

35. Logging Into Fastlane Go to the FastLane website: http://www.fastlane.nsf.gov. Click on Panelist Functions

36. Logging Into Fastlane Use the Log In information provided below to log into Fastlane Panel ID:P1022XX Password: first letter of your first name + first five letters of your last name. You will be prompted to change your password. Trouble Logging in: Note that the ‘0’ is a number, not a letter. If you continue to have trouble logging in, please contact: Phyliss Minn at (703) 292-5087

37. Updating Reviewer Information 1. Click on Edit beside Reviewer Information to update your information 2. Scroll to the bottom of the Reviewer Information screen and enter your banking information. • Logging-in will take you directly to the Panelist System Selection screen • Click on Edit beside Reviewer Information to update your information. Be sure to scroll down to the bottom of the Reviewer Information screen and enter your banking information. This is very important for us to process and send your reimbursement for your time and efforts in a timely manner

38. Requirements • Enter bank information in FastLane for direct deposit by the date specified in the Reviewer’s Guide. • Contingent upon completion of all assigned reviews in FastLane on or before the last day of the panel meeting. • Panelists will receive an IRS 1099 at the end of the year. Reimbursement

39. Accessing Proposals From the Panelist System Selection Screen, scroll to the bottom of the page and click on Panel Review System. • Scroll to the bottom of the Panelist System Selection screen and Click on the Panel Review System to access your proposals.

40. Accessing Proposals Click on Prepare Review Tab. Click on View/ Download Tab. • To view proposals, click on the View/Download tab • To prepare your review, click on the Prepare Review tab

41. Requesting Copies of Proposals 1. Click on Proposal Print Request Tab 3. Click on Print Proposal at NSF or Create CD at NSF 2.Highlight the desired proposals • Scroll down to the bottom of the Proposal Review page, select print proposal at NSF or create CD at NSF. • Click on Print Proposal at NSF or Create CD at NSF to receive a copy of the proposals you were assigned to review in the mail

42. STEP #1 – In FastLane, click on the Prepare Review tab and select the radial button at the top that says “I am giving this proposal no rating.” • STEP #2 - Below the Rating Section, enter your review comments and suggestions in the appropriate intellectual merit, broader impacts, and summary text boxes in the Overall Rating Section • STEP #3 -Your review summarymust include one of the following closing statements, followed by a brief statement of why you are making this recommendation: • “I encourage submission of a full proposal based on …” OR • “I discourage submission of a full proposal due to...” Reviewer Recommendationsin FASTLANE

43. THANK YOU FOR YOUR PARTICIPATION