1 / 52

June 22-25 2009

June 22-25 2009. Reasons for Evaluation:. Basis for improvement of course format/structure Basis for evaluation of effectiveness of teaching methods and impact on learning Justify continued/increased institutional support

ralph-fox
Download Presentation

June 22-25 2009

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. June 22-25 2009

  2. Reasons for Evaluation: Basis for improvement of course format/structure Basis for evaluation of effectiveness of teaching methods and impact on learning Justify continued/increased institutional support Provide supporting evidence for grant proposals for outside funding Use as recruiting/marketing tool for your institution Help us evaluate the effectivenessof the information and resources you’ve gotten from this workshop(FIPSE grant requirement)

  3. Examples of Types of Data to Collect: Percentage of incoming first-year students who place into each level of the targeted courses F/W rates before and after program revision Grade data: average GPA and grade distribution Tutor lab usage Student survey responses

  4. Examples of Reported Data:

  5. Examples of Types of Data to Collect (ct’d): Homework data: % turning it in % score time spent Test/quiz data: Average scores Correlation of scores with Practice test scores Number of practice test attempts Homework completion Class attendance Data on subgroups (e.g. minorities, nontraditional students, male/female, first generation)

  6. More Examples of Reported Data:

  7. Sample data analysis: time spent on homework :

  8. Examples of Types of Data to Collect (ct’d): Costs Start-up Ongoing Estimated savings based on higher pass rate Tuition costs savings to students Instructor FTE savings to university

  9. Data on Program Costs and Savings: A. Startup Costs(first year) • Program Development • Summer salaries for training/course redesign work • Program Director/Administrator • At least 1/4 time, preferably half time for first year, especially if grant-writing is involved • Facilities(mostly a startup cost) • Classroom: Dedicated classroom makes a big difference, especially if equipped with computer access • Tutor lab: Key feature is convenience of location, near classroom and teacher’s offices • Software/Textbooks(for institutions where this cost is not borne by students) • Student Tutors (Our first year total was ~ $60,000, mostly via a special allocation from the Chancellor’s office)

  10. Program Costs: B. Ongoing Costs(subsequent years) • Program Director/Administrator • Minimum ¼ released time during academic year • Summer salary for course revision and evaluation • Student Tutors/TAs • Software Access Codes • NOTE: No new code needed for • Subsequent course with same book • Repeating a course with same book • Miscellaneous • printing, spare calculators, bulletin board, frames, etc.

  11. Program Savings: Immediate savings from decreased failure rates: Using the pre-Math TLC rates, the projected number of drops and withdrawals for Fundamentals of Algebra for Fall 2006 would have been 19 students and for Intermediate Algebra 83 students. The actual number of drops and withdrawals were 4 and 40 respectively, a decrease of 15 students in Fundamentals and 43 in Intermediate Algebra.

  12. Itemized immediate savings from decreased failure rates: • FTE Savings: Based on these numbers the University saved 0.5 teaching position for the Fall 2006 semester. The corresponding salary savings were approximately $8,750 plus $3894 in benefits, for a total of $12,644. • Software Savings: The department saved approximately $2000 in its service and supply budget by not having to purchase access codes to the software used for these courses. • Tuition Savings Students also benefitted from the higher pass rates. Since current tuition is $232.11 per credit, the tuition saved by students during the Fall 2006 semester was $6,963.30 for Fundamentals of Algebra, a two credit course, and $39,922.92 for Intermediate Algebra, a four credit course. Estimated combined annual savings for 2006-07: $120,000

  13. Program Savings: Other Potential Savings and Benefits: • Higher Retention Rates: • Saves tuition dollars • Reduces costs for student recruitment • Program Reputation: • Can serve as a marketing tool for student recruitment, especially in underserved populations, both by official university channels and by word of mouth from satisfied students. • Long-term potential for attracting corporate and alumni financial sponsorship • Basis for Grant Funding: • Proven results from pilot studies make good case for future funding

  14. Longer Term Outcomes Data: Performance in subsequent courses Enrollment inmath next semester Yes/no If yes, which course? Enrollment inschool next semester, next year (retention) Graduation rates, time to graduation Choice of major(esp. STEM vs. non-STEM)

  15. Examples of Longer Term Outcome Data:

  16. Results for Beginning Algebra: More students are passing Math 010 since the start of the Math TLC. The cumulative F/W rate since Fall 04 is 13%, vs. 29% in the four years prior to the Math TLC. This has occurred despite a decline in the math skills of these students coming in to the course, as measured by Math ACT scores.

  17. Combined estimated effect of the Math TLC program over the past 4½ years: ~400 more students passed than predicted vs. the pre-Math TLC F/W rate. This number translates to 3.5% of the entire Stout undergraduate student population over that period, many of whom would likely have dropped out if they hadn’t passed these math classes. Estimated annual savings: • At least ½ FTE teaching position • $75,000-$100,000 in student tuition fees • Reduced student recruitment costs

  18. Some issues in reporting failure or success rates before and after program revision: Define “non-passing”(F? W? D?) Analyze by section, semester, year to show performance progressions over time Length of pre-revision period to report for comparison (We used a 4-year period) Post-revision data: For immediate use: Use section tally forms (4C-1) For final reports: Retrieve official data in same format as pre-revision data

  19. Results to Date – Math 110: 40% drop in F/W rate for 4 years with Math TLC vs. previous 4 years without (1802 total students took Math 110 in the Math TLC in the last 8 semesters)

  20. Sample tally form for final grades completed by each Section Instructor and returned to Course Coordinator at end of semester(Notebook item 4C_1; also on workshop website and CD)

  21. (Instructor names) (Instructor names) (Instructor names) (Instructor names)

  22. (Instructor names) (Instructor names) (Instructor names) (Instructor names)

  23. Sample Tutor Lab Log-in Sheet(Notebook item 4C_2; also on workshop website and CD)

  24. Tutor Lab Use by Day and Week: (Notebook item 4C_3; also on workshop website and CD)

  25. Tutor Lab Use by Course/Instructor: (Instructor names) (Notebook item 4C_3; also on workshop website and CD)

  26. End-of-Semester Student Survey Form(page 1 of 2) (Notebook item 4C_4; also on workshop website and CD)

  27. End-of-Semester Student Survey Form(page 2 of 2)

  28. Student Survey Results:

  29. Comment Card: (Notebook item 4C_5; also on workshop website and CD)

  30. Question list for end-of- semester instructor meeting(page 1 of 2) Ongoing evaluation of teaching methods, course policies, lab procedures (Notebook item 4C_6; also on workshop website and CD)

  31. Question list for end-of- semester instructor meeting(page 2 of 2)

  32. Sample Brief Format Reports: Notebook item 4C9_7 (These reports and others are also on the math TLC website at www.mathtlc.uwstout.edu )

  33. Sample Brief Format Reports: Notebook item 4C_7 (These reports and others are also on the math TLC website at www.mathtlc.uwstout.edu )

  34. Sample Brief Format Reports: Notebook items 4C_8 (These reports and others are also on the math TLC website at www.mathtlc.uwstout.edu )

  35. Sample Brief Format Reports: Notebook items 4C_9 (These reports and others are also on the math TLC website at www.mathtlc.uwstout.edu )

  36. Sample Brief Format Reports: Notebook items 4C_10 (These reports and others are also on the math TLC website at www.mathtlc.uwstout.edu )

  37. Outcomes Survey and Report for Workshop Participants: • Required for our annual report to the FIPSE program that funds these workshops • We will request basic outcomes data from all workshop participants who teach a redesigned course using the methods presented at this workshop. • See the Day 4 “Evaluation” section of your workshop notebook for copies of the outcomes data collection form you will receive near the end of the coming fall semester, plus a copy of the report compiled by our external grant evaluator from the data on redesigned Fall 2007 courses provided by the last year’s workshop participants.

  38. Outcomes Data Collection form sent to 2007 workshop participants (The complete form is in the Day 4 Evaluation Section of your workshop notebook.)

More Related