CAS Standards Program Review and Division-wide Assessment

1. CAS Standards Program Review and Division-wide Assessment

2. With thanks to the generosity ofDeborah A. GarrettVice Chancellor for Student AffairsArkansas State University - Beebe Carole S. Henry, ACUHO-I CAS LiaisonExecutive Director of Housing & Residence Life Old Dominion UniversityDoug Lange, CAS SecretaryDean of Student AffairsAi The Art Institute of Tennessee-Nashville

3. Agenda • Overview of CAS and program review • Preparation for a review • Self Study and Process • Developing an Action Plan • Assessing Student Learning Outcomes • CAS Resources

4. CAS Overview • Founded in 1979 (ACUHO-I charter member) in response to efforts to establish specialized accreditation for student affairs preparation programs • CAS is a consortium of prof. association members (100,000+) in higher education who work collaboratively to promulgate standards & guidelines

5. CAS Overview • Focus on quality programs & services for students • Promulgate standards and guidelines for practice and preparation • Promote assessment in educational practice • Promote quality assurance within higher education

6. WHAT IS PROGRAM REVIEW NOT? • it is not an evaluation of an individual staff member • an evaluation of an individual student • a vehicle for punishment

7. WHAT IS PROGRAM REVIEW? • Formal, collaborative process for systematic review and evaluation • Tied closely to mission, goals, and specified outcomes of a given department • Provides a snapshot of where a department is now • Identifies progress toward department priorities & university strategic planning • Assesses strengths and weaknesses • Makes recommendations for program improvement

8. ELEMENTS OF PROGRAM REVIEW A. Collect, analyze data/evidence about a program B. Compare results to some standard for purpose of evaluating services provided: • effectiveness • efficiency • impact • quality C. Tied to continuous improvement

9. CAS Self Assessment Components • mission • program • leadership • organization and management • human resources • financial resources • facilities, technology, and equipment • legal responsibilities • equity and access • campus and external relations • diversity • ethics • assessment and evaluation

10. CAS Self Assessment Guides Purpose and Organization Outcomes relevant to student learning and development that must be identified and assessed: • knowledge acquisition • cognitive complexity • intrapersonal development • interpersonal competence • humanitarianism and civic engagement’ • practical competence

11. Key Questions for CAS Components • Mission • How does the unit uniquely contribute to and advance the Division’s goals, objectives, & strategic initiatives? • Program • What are the Unit’s core programs and services? • Describe any key collaborative relationships (within and beyond Student Affairs) in which the department is involved. Please explain how these partnerships: • What lines of communication exist for students’ input to be included in the mission/purpose of the dept.?

12. Key Questions for CAS Components • Leadership • What leadership practices are used to establish and promote the unit’s values, short and long term directions, and performance expectations? • How does unit leadership promote a work culture that encourages staff empowerment, innovation, risk taking to effectively meet needs of students? • Philosophy of decision-making? • How does planning take place?

13. Key Questions continued • Organization and Management • Explain the unit’s organizational structure, including reporting relationships, distribution of responsibilities/authority, lines of communication, and use of management and/or work teams. • What criterion does the unit use to accurately determine the correct number of personnel that are needed to effectively fulfill the mission and meet the needs of those it serves?

14. Key Questions continued • Human Resources • What strategies are used to recruit, hire, and retain staff? • How are staff performance expectations established, effectively communicated, and understood? How is staff performance evaluated? How is staff being trained and developed? • Financial Resources • Please provide a financial overview of the unit, • What is the funding model for the unit? What resource changes have occurred over the past five years and what are the anticipated changes for the future? • How are budget allocation/reallocation decisions made in the unit?

15. Key Questions continued • Facilities, Technology, and Equipment • How effectively do current facilities/space and equipment support the work of the unit? • How has technology been integrated into the program, service, and operating functions of the unit? How has the unit kept pace with the development of hardware, software, maintenance, and training support? What are the projected technology needs for the future? • How effectively do the unit’s current facilities, space, and equipment support its operations?

16. Key Questions continued • Legal Responsibilities • How is staff trained to ensure that they are knowledgeable about the laws and regulations, and professional ethics that apply to their respective job responsibilities? • What processes are in place for managing risks • Equity and Access • How is non-discriminatory, fair, and equitable treatment of staff and constituents ensured? • Campus and External Relationships • What collaborative relationships exist with campus colleagues? • What short and long-term goals have been established to strengthen these types of relationships?

17. Key Questions continued • Diversity • How does the unit advance the University and Division’s diversity agenda specific to: • program/service offerings • recruiting, hiring, retaining, and promoting staff • student learning and development • outreach and engagement activities • campus and work environments • Ethics • What is the unit’s policy/strategy for managing student/client and staff member confidentiality issues? How are they informed of these practices?

18. Key Questions continued • Assessment and Evaluation • Provide a description of the unit’s assessment and evaluation activities. • How are student needs assessed and considered in the development and delivery of programs and services? • How is benchmark data used to stay current with peers?

19. Basic Steps:Self-Assessment Process • Establish the process, identify a team and train them. • Understand CAS Standards and Guidelines. • Determine data sources, compile and review evidence. • Gather & analyze pertinent qualitative and quantitative data • Judge performance. • Complete the assessment process.

20. Step B: Understand CAS Standards and Guidelines • CAS functional area standards and guidelines – basic statements that should be achievable by a program • STANDARD (Bold type; Verbs MUST or SHALL) Sample: HLRP staff members must provide a variety of educational opportunities that promote academic success and the achievement of learning and student development outcomes. • GUIDELINE (Regular type; Verbs SHOULD or MAY) Sample: HLRP should provide an environment that assists residents to remain in good academic standing, earn higher GPA’s, and be retained.

21. Step C: Compile and ReviewDocumentary Evidence for Your Area • Recruitment and marketing materials • Participation policies and procedures • Results reports; participant evaluation summaries • Program documents • brochures, catalogs • institutional administrative documents • organization charts • financial statements • Research, assessment, evaluation data • needs assessments, follow-up studies • assessment outcomes reports • Staff activity reports • Student activity reports

22. Step D: Judging Performance • Assessment criterion measures specific to your part of the review ND 1 2 3 4 NR Not Done Not met Minimally Well Fully Met MetMet Using this scale, consider each criterion statement and decide the extent to which each criterion measure has been met by the program or service

23. Step D: Judging Performance continued • Each criterion measure focuses on a particular aspect of the standard, allowing raters to express more detailed and specific judgments • In most instances, there are multiple criteria for each standard

24. Judging Performance Exercise • Page 31: Review must statements • Diversity Criterion Measures • Rate 11.1, 11.2, 11.3, 11.4 independently • Review in group.

25. Formulating an Action Plan • Answer overview questions • Identify areas of program strength (Work form A) • Identify areas of program weakness (Work form A) • Describe practices requiring follow up (Work form A) • Summarize actions required for the program to meet the standards (Work form B) • Summarize program enhancement activities (Work form C) • Write program action plan

26. CAS: Work Form AAssessment, Ratings, and Significant Items CAS Work Form A Assessment, Ratings, and Significant Items INSTRUCTIONS: This work form should be completed following individual ratings of the participants. For each of the 13 Parts, identify (circle) the criterion measure item number(s) in the column labeled for which there is a substantial rating discrepancy (two or more ratings apart). Items not circled should reflect consensus among judges that practice in that area is satisfactory. Items where judgment variance occurs need to be discussed thoroughly by team members. Follow this action by determining which practices (criterion measures) can be designated as “excellent” or “unsatisfactory” and record them in the Step One column. In Step Two, list the items requiring follow-up action including any criterion measure rated as being unsatisfactory by any reviewer.

27. CAS: Work Form A continued

28. CAS Work Form B: Follow Up Actions INSTRUCTIONS: The purpose of this work form is to begin the planning for action to be taken on practices judged to merit follow-up (See Step 1, Work Form A). In Step Three, transfer short descriptions of the practices requiring follow-up and detail these items using the table format provided. Step Three: Describe the current practice that requires change and actions to initiate the change

29. CAS Work Form C:Summary Action Plan Step Four: This form concludes the self-assessment process and calls for action to be taken as a consequence of study results. Write a brief action plan statement in the spaces below for each Part in which action is required.

30. Student Outcomes Assessment • What is the effect of our work on students? • How are they different as a result of interacting with our programs and services? • How do we demonstrate their learning? • How do we know? • How can we show that the outcome was due to our programs, etc.?

31. Plan for Assessing Student Learning Outcomes • Review/revise mission • Identify major services, programs, activities • Specify desired student outcomes • Identify information needed to show student learning • Identify existing and new data needed • Develop process for measuring outcomes • Identify ways to use findings to change/improve

32. CAS Resources • www.cas.edu • CAS Statement of Shared Ethical Principles • CAS Characteristics of Individual Excellence CAS Professional Standards, 2009 edition • Prologue & Context with “How to” information • Self-Assessment Guides SAG’s- tools to help conduct self-assessment activities • Frameworks for Assessing Learning and Development Outcomes (FALDO’s) –contain information for measuring learning outcomes • CD of Self Assessment Guides includes all 38 functional area SAG’s and Standards; also includes an e-learning course on conducting self assessment & a power point presentation that can be modified by user.

33. Program Review Timeline • Self Study (January – April) • Program review committees • students • faculty • staff • Determine adequacy of evidence • Collect more data? • Conduct focus group?

34. Timeline (continued) • Report Findings • Report of findings and recommendations w/emphasis on how unit can improve and contribute more fully.

35. Timeline (continued) • External Review/Site Visit (October) • Two or three external reviewers • Write preliminary report based on Self Study • Campus visit • External Review Team Report

36. Timeline (continued) • Writing the Unit Program Review Report (November/December) • Prepare final report (generally same individuals as self study team) • Share with unit director (ifs/he is not on team) to correct any factual errors • Report to VP, Division Program Review Team, Unit Director, Supervisor

37. Timeline (continued) • Develop the Unit Action Plan (November – December) • Proposed actions • Implementation strategies • Person(s) responsible • Appropriate timelines and identify resources • Include in annual goal-setting