Assessment of Institutional Learning Outcomes (ISLOs): An Integrated Approach

1. Assessment of Institutional Learning Outcomes (ISLOs):An Integrated Approach April 24, 2014 WASC ARC Conference Krista Hall, Program Director / Faculty, General Education, Fresno Campus Megan Lawrence, Sr. Director Accreditation & Licensure Maria Pardorla, Curriculum Manager Randy Roach, Institutional Research Analyst

2. Dialogue over institutional learning outcome assessment data

3. WASC Senior Criteria for Review 4.3 • Leadership at all levels, including faculty, staff, and administration, is committed to improvement based on the results of inquiry, evidence, and evaluation. Assessment of teaching, learning, and the campus environment—in support of academic and co-curricular objectives—is undertaken, used for improvement, and incorporated into institutional planning processes. • Guidelines: The institution has clear, well-established policies and practices—for gathering, analyzing, and interpreting information—that create a culture of evidence and improvement

4. Heald College • Founded in 1863 in San Francisco • WASC Senior Accreditation • 12 campus locations (CA, OR, HI) • Central Administrative Office (CAO) • Over 11,000 enrolled in over 26 programs (2012-2013 IPEDS) • 11% of graduates from 2008-2013 transfer to a four-year university. 23% enroll in a two-year school • Program offerings include two-year degrees in allied health, business, information technology, and legal programs • Planning for baccalaureate degrees

5. Use of ISLO Data at all Levels ( CFR 4.3)

6. Institutional Learning Outcome Assessment

7. Curriculum Support: ISLOs

8. What are the ISLOs? From Academic Catalog…

9. Curriculum Support for Faculty Assessment • ISLO Rubrics (Capstone Courses) • Engage general education faculty in development of common ISLO rubric • Perform quality control • Implement common rubric in all capstone courses • Implement criteria from rubrics in supporting courses • Site Supervisor Evaluations (Internships) • Influenced by programmatic accreditation • Modify as needed based on faculty feedback

10. Curriculum Support: Develop and Implement Assessment Strategy Curriculum Managers . . . • Guide faculty to create curriculum that teach and rubrics that assess attainment of Institutional Learning Outcomes (i.e.:Externship Evaluation form) • Implement common assessment strategies across the twelve campus locations and the online modality

11. Faculty Engagement in the Development of ISLO Rubric 10/19/09 – General Education Program Director Call Curriculum Calls 11/18/09 & 12/17/09 Drafting ISLO Rubric – calls and emails 11/19/09 Announcement of ISLO Rubric in Capstone/Internship Classes as part of 2010 Program Review in the document Program Revision Implementation Plans 1/27/10 & 2/4/10 Revising our draft ISLO rubric for conciseness 8/30/10 Further revising of ISLO Rubric – ILO #4 Initial Discussion of ISLO Rubric for use in Capstone & Internship Courses Agenda Item #1 • Capstones and GE involvement • The curriculum team is exploring possible standard activities and rubrics for assessing institution learning outcomes in the capstone courses across all programs. GE PDs and faculty will be involved in the development of these items.

12. ISLO Rubric - Capstone Courses(Across All Programs)

13. Site Supervisor Evaluation Form (Internship)

14. Assessment Manager SupportISLO data

15. Collect ISLO Data Assessment Project Manager • Timelines • Communications • TracDat Administration, Configuration, Training Curriculum Managers: • Support Faculty / Program Directors Program Directors: • Support Faculty Faculty: • Input Data

16. Faculty Enter Assessment Data into TracDat

17. Institutional Research Support

19. Business AdministrationInstitutional Student Learning Outcomes 2010

20. 2013 Institutional Research provided easy to read graphical representation of ISLO data. Including a Heald-wide comparison of all programs.

21. Institutional Student LearningOutcomes by Program

24. Heald College Institutional Student Learning Outcomes Summary

25. 2012 vs. 2013 ISLO Data

26. Fresno ISLO Data – January 2014 Palooza 2012 2013 Change

27. Potential Questions for Institutional Research to Explore • Do any specific courses or instructors attribute to higher scores? • Are students getting high grades in courses and low outcome scores? • Do scores vary among different student demographics?

28. Program Directors / Faculty

29. Facilitating Assessment Palooza with Faculty Campus Guide Key Elements Enthusiasm Relevance Faculty Buy-In Small Groups • Agenda • Ice breaker • Ground Rules • Clear instructions • Discussion prompts

30. Assessment Palooza Campus Guide

31. Documenting Palooza Results

32. Action Plans

33. Faculty Use of ISLO Data Paralegal - Legal 299 Internship Evaluation: ISLOs that did not meet standard: • Communicates effectively • Evaluates information • Solves problems Action Plan: • Conduct more role play in classes that are conducive (i.e., Legal 215) • Implement plan to help students IRAC more cases in all legal courses • Create problem solving activities for in-class work in all legal courses • Devote entire Legal 299 class sessions to each of the following: effective communication, evaluation of information, and applying critical thinking.

34. Faculty Use of ISLO Data

35. Faculty Use of ISLO Data PHARMACY TECHNOLOGY PHARM 290/296/297 Internship Evaluation: ISLOs that did not meet standard: • Communicates effectively • Evaluates information • Solves problems Action Plan: • Collaborate with English instructors and Gen Ed PD to provide critical thinking/program-related topics for the students. Pharmacy Tech faculty and PD will create topic lists for the various types of papers and presentations, which will give students more practice in pharmacy-related topics that foster critical thinking.

36. Capturing Follow Up from Action Items

37. Campus-Level Change and Improvements Action Item – November 2011 Follow Up – September 2012 Internships are now coordinated by instructors in each major. These instructors consistently devote class time to discussion of their, students’ and employers’ expectations for working in teams. Most instructors cover this topic in class discussions via scenarios often from feedback from site supervisors) for students to work through or roundtable discussions where students bring issues from their sites. For example, our Pharmacy Technology coordinator reinforces 1) the “family-like” atmosphere of most pharmacies and how pharmacy staff rely on one another to ensure patient safety when dispensing the final prescription and 2) importance of networking given because pharmacy is an inter-connected field and Fresno is a small community. ISLO-Collaborates Respectfully At the campus, we need to look at what expectations Students take to their site. Instructors in the classroom need to be explicit about their expectations for working in teams.

38. ISLO Attainment Fresno Campus2012 vs. 2013 Capstone

39. ISLO Attainment Fresno Campus 2012 vs. 2013Internship Supervisor Evaluations

40. Academic Program & Curriculum Changes Institutional Level Examples include: • Curriculum revisions • Addition of courses • Removal of courses • New software, equipment, textbooks, technology • Changes in instructor qualifications • Faculty development and training • Revision of assignments and assessments

41. Tactical Plan Honolulu Campus2013 – 2014

42. ISLOs Honolulu Campus 2013 vs. 2012Capstone

43. ISLOs Honolulu 2013 vs. 2012Internship Supervisor Evaluation

44. Conclusion: Tips for Implementing ISLO Assessment • It’s OK not to be perfect. • Be fearless and try new things… • Clearly define roles • Don’t underestimate the amount of time required to collect student learning samples • Institutional Research support is essential • Create multiple action plans

45. Questions or comments? • Krista_Hall@heald.edu • Megan_Lawrence@heald.edu • Maria_Pardorla@heald.edu • Randall.Roach@heald.edu