CCLI Program Russell Pimmel National Science Foundation rpimmel@nsf Nov 18, 2009

1. CCLI Program Russell Pimmel National Science Foundation rpimmel@nsf.gov Nov 18, 2009

2. CCLI Vision and Scope • Vision: Excellent STEM education for all undergraduate students • Supports efforts that • Bring advances in STEM disciplinary knowledge into curriculum • Create or adapt learning materials and teaching strategies • Develop faculty expertise • Promote widespread implementation of educational innovations • Prepare future K-12 teachers • Enhance our understanding of how students learn STEM topics • Enhance our understanding how faculty adopt instructional approaches • Build capacity for assessment and evaluation • Further the work of the program itself

3. CCLI Vision and Scope (Cont) • Program especially encourages projects that: • Have potential to transform undergraduate STEM education • Produce widespread adoption of classroom practices based on how students learn • Explore cyberlearning

4. PROJECT COMPONENTS • Creating Learning Materials and Strategies • Incorporate ideas from research on teaching and learning • Incorporate scientific advances in disciplines • Implementing New Instructional Strategies • Implement proven or promising techniques in ways that encourage widespread adoption • Add knowledge about transfer of strategies and impact on learning • Developing Faculty Expertise • Increase instructor’s knowledge and skills on curricula and teaching practices • Involve diverse group of faculty

5. PROJECT COMPONENTS (cont) • Assessing and Evaluating Student Achievement • Develop and disseminate valid and reliable tests of STEM knowledge and skills • Collect, synthesize, and interpret information about students • Conducting Research on Undergraduate STEM Education • Explore how undergraduate STEM students learn • Explore how practices have diffused and how faculty and programs implement changes • NOTE: Instrumentation and equipment requests are appropriate -- based on learning impact

6. Important Project Features • Quality, Relevance, and Impact • Describe a recognized need or opportunity and an innovative approach • Student Focus • Link activities and improvements in STEM learning • Knowledge about STEM Education: • Build on existing work & disseminate new finding and results • STEM Education Community-Building • Interact with others in the STEM education community • Sustainability • Demonstrate reasonable expectation of persistent effects • Expected Measurable Outcomes • Describe goals & expected measurable outcomes • Project Evaluation • Monitor progress toward expected outcomes and success in achieving them

7. Type 1 Projects • 70 to 75 awards expected • Total budget up to $200,000 for 2 to 3 years • 250,000 when 4-year and 2-year schools collaborate • Deadline • May 21, 2009 (A-M states) • May 22, 2009 (N-Z states) • Typically involve a single institution & one program component • Contribute to the understanding of undergraduate STEM education

8. Type 2 Projects • 20 to 25 awards expected • Total budget up to $600,000 for 2 to 4 years. • Deadline January 13, 2010 • Typically involve multiple institutions & several program components – but exceptions • Typically based on prior work with results explicitly described – but exceptions • Produce evidence on the effectiveness • Institutionalize at the participating schools

9. Type 3 Projects • 3 to 5 awards expected • Budget negotiable, but not to exceed $5,000,000 over 5 years. • Deadline January 13, 2010 • Large scale efforts • Typically based on prior work with results explicitly described – but exceptions • Produce evidence of student learning in a broad population • Describe impact of the work on the prevailing models • Describe strategies for implementation in new contexts

10. CCLI Central Resource Projects • 1 to 3 awards expected • Budget negotiable, depending on the scope and scale of the activity • Small focused workshop projects -- 1 to 2 years & up to $100,000 • Large scale projects -- 3 to 5 years & $300,000 to $3,000,000 • Deadline January 13, 2010 • Implement activities to sustain the STEM community • Increase the capabilities of and communications in the STEM community • Increase and document the impact of CCLI projects

11. NSF Merit Review Criteria • Standard intellectual merit and the broader impacts review criteria • Additional intellectual merit review criteria • Produce exemplary material, processes, or models that enhance student learning • Yield important findings related to student learning • Build on existing knowledge about STEM education • State expected outcomes & integrate them into evaluation plan • Describe evaluation plan that is likely to produce useful information • Additional broader impacts review criteria • Contribute to the understanding of STEM education • Help build the STEM education community • Have a broad impact on recognized need or opportunity • Have the potential to contribute to transformative change

12. CCLI Program – Information Sites • Solicitation http://www.nsf.gov/funding/pgm_summ.jsp?pims_id=5741&org=DUE&from=home • Search awards http://www.nsf.gov/awardsearch/tab.do?dispatch=2 • Use “Search All Fields” tab • Enter key words • Enter “Element Code” -- use “Lookup” link on right • Select “Any” vs “All” • Can request copy of proposal from PI or NSF (FOIA) http://www.nsf.gov/policies/foia.jsp • Use carefully – Not a “template” for your idea

13. Questions

14. Common Strengths and Weaknesses in CCLI Proposals

15. CCLI Review Processes • Program directors • Sort by disciplines • Send to reviewers • Each reviewer • Rates each proposal (E, V, G, F, and P) • Writes comments • Describes strengths and weaknesses in terms of the intellectual merit and broader impacts criteria • Review panel • Discusses each proposal • Writes the Panel Summary • Highlights strengths and weaknesses

16. Study of Strengths and Weaknesses • Analyzed strengths and weaknesses identified in Panel Summaries of CCLI Phase 1 engineering proposals from 2005 and 2006 • Developed codebook of statements (or items) describing strengths and weaknesses • Included 30 complementary strength and weakness statements, e. g., “Proposal was innovative” and “Proposal was not innovative” • Coded Panel Summaries for 471 proposals

17. ActivityStrengths & Weaknesses • Pretend you analyzed a stack of panel summaries to identify the most commonly cited strengths and weaknesses • List what you think will be • Most common strengths (Proposal was innovative) • Most common weaknesses (Proposal was not innovative) Predict the results of our analysis

18. Most Common Strengths

19. Most Common Weaknesses

20. Top Ten Strengths and Weaknesses

21. Most Commonly Cited Strengths and Weaknesses

22. Questions