Enhancing Engineering Education Proposals Workshop Notes

1. Writing More Effective Proposals Russ Pimmel Abe Nisanci U of Alabama NSF. Share The Future IV March 17, 2003

2. Workshop Format • “Working” workshop • ½ to ¾ of time in team activities • Limited time to complete activities • Frequently feel you need more time • Purpose is to get you started • No “answers” or even the “formulas” • Purpose: identify, consider, & discuss ideas

3. Workshop Goals • Participants should be able to: • List areas where good engineering education proposals can be improved • Generate a list of suggestions for each area

4. Beyond a Good Idea • Proposals must describe a good idea • It must be explained and developed • Workshop will assume a good idea • Focus on areas for enhancing a proposal that contains a good idea

5. Warning on Generalizations • NSF has several programs supporting undergraduate education • Different requirements • Different slants • Proposal improvement ideas apply to all -- but in varying degrees • Choose ideas based on • Program solicitation • Judgment

6. Scenario – Developing a Proposal Idea • Prof. ____ has taught ENG ___ for several semesters • He has idea for “greatly improving” the course by adding new stuff • new stuff = laboratory, web experience, interactive set of material, workbook , new text • He tried some preliminary material • Based on this, Prof. ____ decided to prepare an NSF proposal

7. Proposal Skeleton • Goal: Develop new stuff to enhance student learning at U of _____ • Rationale: Observed shortcomings in educational experience of the students at U ____ & felt that new stuff would improve the situation • Project Plan: “Details of new stuff “ • Evaluation: Conduct course evaluations when using new stuff • Dissemination: Describe new stuff using conference, journal papers, and web site

8. What’s Wrong? • TASK: • Prepare a list of ideas for improving this proposal • What advice would you give Prof ___ • PROCESS: • Individually prepare a list • Share ideas with neighbor • Report neighbor’s best idea

9. What’s Wrong? -- NSF Project Directors’ Responses – Goals • Indicate the development, evaluation, and assessment are the real goals • Not “enhanced learning of students at U of ____”

10. What’s Wrong? -- NSF PD’s Responses – Rationale • Describe experience at other schools • Reference the educational literature • Discuss effects on retention and broader participation • Indicate why approach is new and innovative

11. What’s Wrong? -- NSF PD’s Responses – Evaluation • Use external evaluator or assessment expert • Include collaboration with other faculty at same or different school • Include beta test at another site • Include measures of student learning in evaluation process • Tie evaluation to goals and objectives • Include impact statement

12. What’s Wrong? – PD’s Responses – Dissemination • Include collaboration with faculty members in other schools • Include an outreach component • K-12 or community colleges • Include beta testing at other school • Include faculty workshop

13. What’s Wrong? -- PD’s Responses – General • Include letters of support • Form a collaborative effort • Include a plan with timeline, milestones, and responsibilities • Make sure to select the appropriate NSF program

14. What’s Wrong – Four Concerns • Goals focused on local problem • Ignores broader impact • Considers only applicant’s experience • Ignores the experience of others • Ignores the literature • Limits evaluation to students’ impressions • Ignores learning goals and outcomes • Dissemination plan passive • Needs to be proactive and aggressive

15. What’s Wrong – Four Concerns • Goals focused on local problem • Ignores broader impact • Considers only applicant’s experience • Ignores the experience of others • Ignores the literature • Limits evaluation to students’ impressions • Ignores learning goals and outcomes • Dissemination plan passive • Needs to be proactive and aggressive

16. Improving Rationale -- Global vs Local Problem • TASK: • Generate a list of locations or sources that will provide a broader view of the problem leading to broader goals • PROCESS: • Individually prepare a list • Work as a team to build a consensus list • Report team’s ideas

17. Strategies Team Exercises • Be positive, supportive, and cooperative • Limit critical or negative comments • Be brief and concise in discussions • Avoid lengthy comments, stories • Avoid arguments • Stay focused • Designate roles • Coordinator, recorder, gatekeeper

18. NSF PD’s Responses --Global vs Local Problem • Education literature • Journals and conference proceeding • Education sessions at discipline meetings • Lay scientific press • NY Times science section • Panel reports • “What’s wrong with Education in _____’ • Industry or advisory board input

19. NSF PD’s Responses --Global vs Local – Part 2 • NSF web site • Education oriented web sites • Teaching and learning centers at some universities • Education pages at professional society sites • Colleagues at other schools • Web sites at other schools

20. Improving Goals & Objectives Statement • TASK: • Generate a list of improvements for the goals and objectives in Reading # 1 • A list of suggestions that will broaden and clarify the goals and objectives • PROCESS: • Individually prepare a list • Work as a team to build a consensus list • Report team’s ideas

21. PD’s Responses – Improving Goals & Objectives • Relate goals to student learning objectives • Use more specific, goal-oriented verbs • “Enhance “ and “acquaint” are vague • Don’t describe measurable actions • Be more specific • Eliminate the “apple pie” goals

22. PD’s Responses – Improving Goals & Objectives – Part 2 • Use broader goals • Don’t just focus on effects on student's in PI’s course • Make the goals to develop, evaluate, and disseminate material • Be careful about the distinction between goals and objectives • Goals – higher-level, broad-reaching • Objectives – specific, measurable outcomes

23. What’s Wrong – Four Concerns • Goals focused on local problem • Ignores broader impact • Considers only applicant’s experience • Ignores the experience of others • Ignores the literature • Limits evaluation to students’ impressions • Ignores learning goals and outcomes • Dissemination plan passive • Needs to be proactive and aggressive

24. Build on Experiences of Others • TASK: • Generate a list of locations or sources that describe prior work by others • PROCESS: • Individually prepare a list • Work as a team to build a consensus list • Report team’s ideas

25. NSF PD’s Responses -- Others’ Experiences • Same as earlier list • These sources • Justify a broader need • Summarize others’ experiences

26. Improving Rationale • TASK: • Generate a list of improvements for the rationale statement in Reading # 2 • A list of suggestions that will provide a broader view • PROCESS: • Individually prepare a list • Work as a team to build a consensus list • Report team’s ideas

27. PD’s Responses – Improving Rationale • Discuss shortcomings or problems in programs described by others • Discuss the general need for the new material -- the need at other schools • Indicate student interests (current & projected) – include references • Indicate demand for graduates (current & projected) – include references • Discuss how the new material will fit in the existing curriculum

28. What’s Wrong – Four Concerns • Goals focused on local problem • Ignores broader impact • Considers only applicant’s experience • Ignores the experience of others • Ignores the literature • Limits evaluation to students’ impressions • Ignores learning goals and outcomes • Dissemination plan passive • Needs to be proactive and aggressive

29. Evaluate Goals, Implementation, Outcomes • TASK: • Generate a list of aspects that can be evaluated in a projects • PROCESS: • Individually prepare a list • Work as a team to build a consensus list • Report team’s ideas

30. NSF PD’s Responses –Evaluation Aspects • Measure gains in student learning • Pre and post tests • Experimental and control groups • Use formative and summative evaluations • Formative to guide development • Summative to verify & document success • Include diverse audiences • Universities & community colleges • Majors and non majors

31. NSF PD’s Responses –Evaluation Aspects – Part 2 • Evaluate at several levels • Appropriateness of learning objectives • What is being taught/learned • Attitude of students • How is it being taught • Learning outcomes • How successful was the instruction • Examine effects on retention and diversity • Consider beta testing

32. Evaluate Goals, Implementation, Outcomes TASK: • Generate a list of improvements for the evaluation plan in Reading # 3 • PROCESS: • Individually prepare a list • Work as a team to build a consensus list • Report team’s ideas

33. PD’s Responses – Improving Evaluation • Add more formative evaluation • Monitor students’ attitude and learning during course • Measure student learning • Need learning objectives • Include copy of evaluation tool or sample questions • e. g., student survey form • Develop specific criteria for evaluation by other faculty in subsequent courses

34. What’s Wrong – Four Concerns • Goals focused on local problem • Ignores broader impact • Considers only applicant’s experience • Ignores the experience of others • Ignores the literature • Limits evaluation to students’ impressions • Ignores learning goals and outcomes • Dissemination plan passive • Needs to be proactive and aggressive

35. Use Active, Aggressive Dissemination • TASK: • Generate a list of approaches for disseminating results of project • PROCESS: • Individually prepare a list • Work as a team to build a consensus list • Report team’s ideas

36. NSF PD’s Responses – Dissemination Approaches • Educational journals & meetings • Don’t neglect regional meetings • Faculty workshops • Personal or course web sites • Professional group or subspecialty web sites • Professional group or subspecialty newsletters

37. NSF PD’s Responses – Dissemination – Part 2 • Textbooks, manuals, instructor guides • Agreements with other faculty members to critique or evaluate material • Mailing to colleagues • General or targeted

38. Include Active, Aggressive Dissemination • TASK: • Generate a list of improvements for the dissemination plan in Reading # 4 • PROCESS: • Individually prepare a list • Work as a team to build a consensus list • Report team’s ideas

39. PD’s Responses – Improving Dissemination • Any or all items on previous list

40. Practical Aspects of Review Process • Reviewers have ten or so proposals from several areas • Reviewers have limited time for your proposal • Reviewers may be experienced or inexperienced in review process • Reviewers may be an expert or a novice in proposal area

41. Dealing With Practical Aspects of Review Process • TASK: • Generate a list of approaches that an applicant should consider in dealing with these practical aspects • PROCESS: • Individually prepare a list • Share ideas with neighbor • Report team’s ideas

42. NSF PD’s Responses –Practical Aspects of Review • Use good style • Be concise, specific, but complete • Write simply but professionally • Avoid jargon • Use spell and grammar checkers

43. NSF PD’s Responses –Practical Aspects – Part 2 • Follow guidelines • Double space, use correct font size • Use readable structure • Use sections, headings, bullets • Follow the order given in solicitation • Use appendices sparingly

44. NSF PD’s Responses –Practical Aspects -- Part 3 • Emphasize results when writing about prior funding • Reinforce your ideas • Summarize • Highlight them (e. g., use bullets) • Pay attention to the rationale, goals and objectives, evaluation, and dissemination • Have expert and non-expert read draft version

45. Warning on Generalizations • NSF has several educational programs • Different requirements & slants • Proposal improvement ideas apply to all -- but in varying degrees • Read the solicitation carefully

46. Beyond a Good Idea • To enhance a good proposal • Describe broader impact in rationale & goals • Consider and build on others’ experiences • Cite the literature • Evaluate learning goals, students’ impressions, outcomes, etc. • Include proactive & aggressive dissemination

47. Use Judgment • When writing proposals, you will wonder • “Should I include ____?” • “Should I do _____?” • “How should I do _____?” • The answer is “It depends.” • There is no magic formula. • Read the solicitation • Use your judgment • Don’t include a half-bake section because someone told you that it’s you needed

48. Final Comment If you have a good idea, thinking about it in terms of • How you could broaden the objective • How you could relate it to the literature • How you could evaluate it • How you could interest others in it will sharpen the idea

49. Questions

50. Russ Pimmel rpimmel@coe.eng.ua.edu 205-348-1753 Abe Nisanci inisanci@nsf.gov 703-292-4644