1 / 29

Program Assessment Workshop for Staff A Presentation for Palo Alto College 1/10/2018

Program Assessment Workshop for Staff A Presentation for Palo Alto College 1/10/2018. Keston Fulcher, PhD James Madison University. What is Assessment?. Assessment is a bureaucratic exercise that has little bearing to our work. OR…

salinas
Download Presentation

Program Assessment Workshop for Staff A Presentation for Palo Alto College 1/10/2018

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Program Assessment Workshop for StaffA Presentation for Palo Alto College1/10/2018 Keston Fulcher, PhD James Madison University

  2. What is Assessment? • Assessment is a bureaucratic exercise that has little bearing to our work. OR… • Assessment is a process that compellingly tells our story: The goals we’ve set; the strategies we’ve used to accomplish them; how we’ve affected our students and institution; how we’ve learned from our failures; how we’ve celebrated our successes.

  3. A Note about Improvement • The Damon Dash Story….

  4. Overview • Admin vs. Academic Program Asmt • Outcomes for Participants • Contrast outcomes and actions in an assessment framework • Distinguish among beginning, developing, good, and advanced assessment reporting • Discuss the fundamentals of identifying problems, collecting baseline data, systematically addressing the issues, and then re-assessing to determine success

  5. Admin vs. Academic Asmt • Academic Units: Focus typically on student learning outcomes • Administrative Units: Focus typically on effectiveness/efficiency (more on this later) • Administrative Units: Much more diverse

  6. Outcome vs. Action • (Intended) Outcome: A desired IMPACT of your office’s efforts • Action: What you do to achieve outcome • The two are often confused. However, sometimes the distinction is not clear cut.

  7. Example 1 • Action Two pills every four hours for three days • Intended Outcome My temperature will drop from 101 to 98.6 Cause - Effect

  8. Example 2 • Action I will • exercise 30 minutes four days a week and • eat approximately 2000 calories a day • Intended Outcome I will lose 20 pounds Cause - Effect

  9. Example 3 • Action • Realtors will attend summer retreat on better sales tactics. • Realtors, on average, will show 50 houses per year. • The marketing office will advertise via Google, the weekly real estate digest, etc. • Intended Outcome The real estate office will sell 10 million dollars worth of homes in 2018 (2 million more than 2017) Cause - Effect

  10. Outcome or Action? • The maintenance crew will keep the lawn mowed.

  11. Outcome or Action? • The assessment director will meet with all unit coordinators on campus.

  12. Example 4 (A PAC Flashback) The year, 2008 • Top Song: https://www.youtube.com/watch?v=dq6Q_uaJF4k • Top Movie: https://www.youtube.com/watch?v=VNKNQwJIlBc

  13. Top News Story 2008 Image From: http://www.businessjournalng.com/case-study-the-collapse-of-lehman-brothers/

  14. 2008 PAC FT 4-yr FTIC Graduation Rate 11.2%

  15. Segue to Outcome/Action • (Intended) Outcome: The 2012 FT PAC cohort will have a 4-yr FTIC graduation rate of 22.4%. Note, 2008 rate was 11.2% • What actions did you take to get here….

  16. In Business Parlance • Outcomes = LAG measures • Actions = LEAD measures

  17. The Assessment Cycle

  18. Step 1: State Service-Level Outcome Service-Level Outcomes (SLOs) are the desired effect your unit/program wishes to have.

  19. Step 2: Map Actionsto Outcomes Identify actions you intend to take to achieve desired outcome Referred to as Program Theory • How the design of a program should theoretically affect outcomes

  20. Step 3: Select Methods Select instruments and data collection strategies to gather evidence about outcomes.

  21. Step 4: Analyze and Interpret Results What did you find, and what does it mean relative to your outcomes and actions?

  22. Step 5: Reporting to Stakeholders Identify stakeholders and what they want and need to know.

  23. Step 6: Using Results for Improvement Program/unit improvement based on evidence is the main purpose of assessment

  24. O.k….Let’s Pivot to Assessment Reporting • Four hypothetical assessment reports; handing out one at a time; each progressively better • Report format inspired after PAC’s • Purpose: Aligned with…Distinguishing among beginning, developing, good, and advanced assessment reporting • Context: • Unit: The Health Center • Outcome: Flu shots ACTIVITY

  25. On Learning Improvement A Pig Never Fattened Because It was Weighed

  26. Fundamental Considerations of Improvement • It’s about relationships and trust • Leadership within the unit • Clarity of shared vision regarding improvement • Tight relationship between actions and outcomes • Development of staff • Assessment built around (pre and post) actions • Contextual factors

  27. Practice Scenario… • Pretend you are an improvement consultant The director of the Health Center approaches you… • You are trying to size up the program and its readiness for an improvement project. • What questions should you ask? • For example, what do you want to improve? • Work with your partner to generate questions. • 10 minutes. • Let’s role play ACTIVITY

  28. Summary and Questions • Outcomes for Participants • Contrast outcomes and actions in an assessment framework • Distinguish among beginning, developing, good, and advanced assessment reporting • Discuss the fundamentals of identifying problems, collecting baseline data, systematically addressing the issues, and then re-assessing to determine success

  29. fulchekh@jmu.edu • References • First Pig Paper: http://learningoutcomesassessment.org/documents/Occasional_Paper_23.pdf • Second Pig Paper: http://www.rpajournal.com/return-of-the-pig-standards-for-learning-improvement/

More Related