1 / 24

Evaluation

Evaluation. EnAct Campuses Sonoma, Humboldt, Chico, San Francisco, San Jose, Fresno, Bakersfield, Pomona Presentation July 11, 2007 Public Works , Inc. Mikala L. Rahn, PhD Patty O’Driscoll, MPA. Who Are We?.

micheal
Download Presentation

Evaluation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Evaluation EnAct Campuses Sonoma, Humboldt, Chico, San Francisco, San Jose, Fresno, Bakersfield, Pomona Presentation July 11, 2007 Public Works, Inc. Mikala L. Rahn, PhD Patty O’Driscoll, MPA

  2. Who Are We? Public Works, Inc. a nonprofit in Pasadena dedicated to working with communities, government, schools and parents by providing services and resources to educate and inform children, youth and families. Our work is in three areas: • Education Reform • Workforce Development • Intervention/Prevention

  3. Evaluation Goals • Measure the success of participating faculty and their students with disabilities overtime as well as compared to students without disabilities on a number of measures • Monitor the project based on federal requirements and offer program improvement strategies  Improve the consistency of measurement across the institutions

  4. Research Questions 1) Which of the 14 elements of the Universal Design for Learning (UDL) are most important or are they all equal? 2) How does faculty training in UDL impact student resilience (persistence/graduation)? Are there more appropriate courses or faculty to target to affect resilience?

  5. Research Questions 3)Why do faculty participate? 4)How is the use of online and other technologies cost-effective, beneficial, and scalable for rural and remote regions serving the target populations? 5)Did the project reach the target segments of the CSU system with information about research and developments in effective teaching of students with disabilities?

  6. Balanced Evaluation Approach Process measures focus on how a program is implemented and the outcome measures focus on the results of the program or intervention. Public Works, Inc. is using both quantitative and qualitative data collection methods

  7. Methodology Baseline data from participating institutions; annual data collection Attendance data at workshops and institutes Preliminary Assessment of Faculty Implementation of UDL UDL Training Evaluation • Faculty Activity Survey/Assessment Course Artifacts • Web user statistics

  8. Federal Grant Performance Report (GPR) Objectives…Measures...Targets...Performance Data • The difference between the rate at which students with documented disabilities complete courses taught by faculty trained in project activities, and the rate at which other students complete those courses. • The percentage of faculty trained in project activities that incorporate elements of training into their classroom teaching.

  9. Federal Grant Performance Report (GPR) Objectives…Measures...Targets...Performance Data • Collaboration to Ensure Access to a Quality Postsecondary Education • Technology to Ensure Access to a Quality Postsecondary Education • Dissemination of EnACT Content and Processes

  10. Program Improvement How do we better serve students with disabilities? How do we better support faculty? What do we need to know and be able to do?

  11. Overall Evaluation Goals • Keep the grant by meeting the requirements • Learn lessons from each other • Support faculty and students

  12. Summary of ResultsAnnual Performance Report Goal 1: Collaboration • EnACT Leadership Team, Advisory Committee, and campus-based Communities of Practice established • CSU Fresno, Bakersfield and Pomona to join current collaborative • CSU ATI Initiative partnership at 2007 Institute • UDL workshops attended by 194 attendees-target was 100

  13. Summary of ResultsAnnual Performance Report Goal 2: Technology to ensure access • Enhancement of AIM resource specifications for authoring • 13 AIM resources proposed or published to date

  14. Summary of ResultsAnnual Performance Report Goal 3: Dissemination of EnACT • Enhancements of Web site • Dissemination at 11 professional conferences or Webcasts

  15. Summary of ResultsAnnual Performance Report Highlights Project Specific Performance Measures: Course Completion Rate • Students with disabilities completed at 100% (increase of 7% from baseline) n=9 • Students without disabilities completed at 96% n=328 • GPA for students with disabilities was 3.22 (increase of .22 from baseline) • GPA for students without disabilities was 3.26

  16. Summary of ResultsAnnual Performance Report Highlights Project Specific Performance Measures: Faculty participation • Faculty participation across campuses totaled 34, above target of 25 • 73% of faculty agreed they would “more likely make changes to their courses” after exposure to UDL workshops (target was 60%) • All participating faculty reported making changes to their courses as a result of participation

  17. Summary of ResultsFaculty Interview Highlights Changes to Courses • Changes the result of the combination of training they received, not individual components • Changes ranged from revisions to teaching strategies to more technologically complex (multi-media, Web CT) • Prompted by desire to make course accessible to all students rather than for accommodations to individual students

  18. Summary of ResultsFaculty Interview Highlights Sample of course changes • Students acting out lessons or concepts • Improvements to posting on the Internet • Revisions to Syllabi using EnACT info • Revising PowerPoint to be more accessible • Study guides, guided lecture notes, assigning all students to be note takers • Using rubrics and adding new assessment strategies

  19. Summary of ResultsFaculty Interview Highlights Faculty Feedback about EnACT • Each component received positive feedback: overview from UDL I/II, specific information from Institute, and collaboration in Faculty Learning Community • Changes to courses had positive impact on teaching and learning; support from EnACT prepared them to continue to make changes

  20. Summary of ResultsFaculty Interview Highlights Faculty Feedback about EnACT • Would like support in implementation of technology • Would like continued opportunities for collaboration that occurred during FLC

  21. Evaluation Recommendations • Continue to examine ways to implement EnACT to impact larger numbers of students with disabilities • Use lessons learned from EnACT to implement ATI initiative and expand impact on campus • Examine the impact, use and usefulness of AIMs

  22. Evaluation Next Steps • Orient new campuses to evaluation requirements at Leadership meeting • Work with campuses to continue to streamline data collection processes • Survey students in Fall 2007 • Collect Cohort II implementation data in Spring 2008 (Fall 2007 grades) • Collect Cohort III baseline data in Spring 2008 (Fall 2007 grades)

  23. Contact Us Mikala L. Rahn, PhD President Public Works, Inc. 90 N. Daisy Avenue Pasadena, CA 91017 (626) 564-9890 (626) 564-0657 mrahn@publicworksinc.org

  24. Contact Us Patty O’Driscoll Consultant Public Works, Inc. 1191 Loma Ct. Sonoma, CA 95476 (707) 933-8219 (707) 996-8726 fax patty@publicworksinc.org

More Related