1 / 50

Non-Academic Assessment: Completing an Assessment Report

Non-Academic Assessment: Completing an Assessment Report. Office of Assessment July 12, 2012. Goals of Workshop. Current assessment context What’s new this year Introduce new Higher Learning Commission Criteria Review items on the Assessment Report template

karina-holden
Download Presentation

Non-Academic Assessment: Completing an Assessment Report

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Non-Academic Assessment: Completing an Assessment Report Office of Assessment July 12, 2012

  2. Goals of Workshop • Current assessment context • What’s new this year • Introduce new Higher Learning Commission Criteria • Review items on the Assessment Report template • Walk through new online reporting form

  3. Assessment Context

  4. Assessment Context Assessment Context • Last year participating in the Higher Learning Commission’s (HLC) Academy for the Assessment of Student Learning • New HLC Criteria effective January 1, 2013 • Campus focus on 2020 Strategic Plan alignment Assessment Reporting Themes • Focus on use of data • Reflect on changes and impact on learning • Streamline reporting process Future Goals • Longer reporting cycle • Integration with annual reporting

  5. What’s New This Year • Online reporting • Fewer questions • Questions “tagged” with New HLC Criteria • Optional question on New HLC Criteria • Report quality threshold to 70%

  6. HLC New Criteria • Guiding Values, includes: 1. Focus on student learning 4. Culture of continuous improvement 5. Evidence-based institutional learning and self-presentation 9. Mission-centered evaluation • Anatomy of Criteria • Criterion—broad statement • Core Component—areas of particular focus • Sub-Component—further elaboration on Core Component, but not exhaustive (and not always included) • Some criteria now specifically mention certain units (e.g., advising)

  7. HLC Criteria • HLC Criterion One. Mission The institution’s mission is clear and articulated publicly; it guides the institution’s operations. Core Component 1.A. The institution’s mission is broadly understood within the institution and guides its operations. • Subcomponent 2. The institution’s academic programs, student support services, and enrollment profile are consistent with its stated mission. Criterion Title and broad statement Core Component statement—particular aspect of criterion on which to focus Sub-component—specific area from which evidence is expected

  8. HLC New Criteria • The Five Criteria • Mission • Integrity: Ethical and Responsible Conduct • Teaching and Learning: Quality, Resources, and Support • Teaching and Learning: Evaluation and Improvement • Resources, Planning, and Institutional Effectiveness • Assumed Practices • Won’t check, but we are responsible to ensure/remediate • Notable is B.2.c.1): Faculty participate substantially in: analysis of data and appropriate action on assessment of student learning and program completion.

  9. Upcoming Assessment Workshops Developing an Assessment Plan Thursday, July 19th, 10-11 am Completing the Assessment Report Tuesday, July 17th, 1-2 pm Note: workshops/forms overlap

  10. Completing an Assessment Report

  11. Unit Information • Explain how your unit supports 2020 Strategic Plan of YSU. • Removed mission; strategic plan operationalizes mission • HLC Criterion One. Mission • The institution’s mission is clear and articulated publicly; it guides the institution’s operations. • Core Component 1.A. The institution’s mission is broadly understood within the institution and guides its operations. • Sub-component 2. The institution’s academic programs, student support services, and enrollment profile are consistent with its stated mission. • HLC Criterion Three. Teaching and Learning: Quality, Resources, and Support • The institution provides high quality education, wherever and however its offerings are delivered. • Core Component 3.E.The institution fulfills the claims it makes for an enriched educational environment. • Sub-component 1. Co-curricular programs are suited to the institution’s mission and contribute to the educational experience of its students.

  12. Example: Strategic Plan Link YSU’s 2020 Strategic Plan Office of Assessment (OOA) The new University-wide Strategic Plan is organized around four cornerstones • Student success • The urban research university mission/transition • Regional impact • Institutional accountability and sustainability Two goals of the OOA (to increase campus engagement in assessment and to successfully demonstrate progress in the HLC Academy for the Assessment of Student Learning) are directly tied to 2 cornerstones of the 2020 Plan Accountability and Sustainability which emphasizes shared responsibility. Student Success since it documents student achievement in curricular and co-curricular areas. The new Engagement in Assessment Portfolio will provide documentation.

  13. Example: Strategic Plan Link YSU’s 2020 Strategic Plan Office of Community Involvement Theme 2.2. Satisfaction with non-academic experiences and campus life-including the importance of participation in service learning and community activities The main goal of this unit is to increase participation in community activities as well as meaningful reflection on the impact of those activities on students’ development

  14. Unit Information • Describe the goals for your unit from the previous academic/fiscal year Example (from Office of Campus Planning and Community Partnerships): • to annually update the Campus Master Plan and undertake such studies as may be required to implement specific recommendations in accordance with the University’s annual capital improvement program; • to complete the Campus Roadway Study initiated by the City of Youngstown and focused upon the major arterial roadways serving the campus; • to implement improved bus service to the campus in collaboration with the WRTA and the University’s Alternative Transportation Community; and • to collaborate with Student Government to develop a matrix of student volunteer requirements and opportunities.

  15. Learning Outcome Focus 3. State at least one learning outcome that is relevant for your unit • Keep to 1 (maybe 2 LO’s) • Keep to 1 focused project (not a laundry list) • What are you interested in? • HLC Criterion Four: Teaching and Learning: Evaluation and Improvement • The institution demonstrates responsibility for the quality of its educational programs, learning environments, and support services, and it evaluates their effectiveness for student learning through processes designed to promote continuous improvement. • Core Component 4.B. The institution demonstrates a commitment to educational achievement and improvement through ongoing assessment of student learning. • Sub-component 1. The institution has clearly stated goals for student learning and effective processes for assessment of student learning and achievement of learning goals.

  16. Developing Learning Outcomes Sources for LO: • A particular project that your unit will be focusing on in 2011-12 OR • LO that staff from your unit develop OR • YSU’s General Education learning outcomes OR • The Council for the Advancement of Standards in Higher Education (CAS) General Standards OR • Learning Reconsidered

  17. Developing Learning Outcomes Developing a learning outcome: • Framing Question: What do you want students/staff/faculty to know and be able to do as a result of interacting with your office or program you offer? • Should be SMART: • Specific, Measurable, Attainable, Realistic, Timely • Two common methods to writing a learning outcome: • ABCD Model • SVP Model Learning outcome material excerpted from GLACUHO Webinar on Writing Learning Outcomes by Jeff Long, May 2010

  18. Developing Learning Outcomes ABCD Model of Learning Outcomes Audience-target of the outcome Behavior-verb in future tense Condition-activity causing learning Degree of Achievement (optional) ________ ________ ________ ________ Condition Audience Behavior Degree

  19. Developing Learning Outcomes ABCD Model Examples ________ ________ ________ ________ Condition Audience Behavior (Degree-optional) • As a result of student staff training, student staff will be able to list (4 of the 5) department’s values. • Through publicity and training, staff will correctly complete online timesheets (90% of the time). • As a result of this workshop, attendees will explain the components of a well-written assessment report.

  20. Developing Learning Outcomes SVO Model of Learning Outcomes Subject = learner Verb = cognitive process Object = content/knowledge Note: removes condition (e.g. specific intervention); somewhat more like academic learning outcomes. Examples: • Students will identify academic resources on campus. • Faculty will use DARS system in advising.

  21. Developing Learning Outcomes Using Verbs in Learning Outcomes • Avoid “know,” “understand,” “think,” etc. –how will you measure? • Consider the level of learning you expect from your students/staff/faculty—do you want them to comprehend? Apply knowledge? • Verb list that considers the level of learning (aka Bloom’s Taxonomy)--very helpful!: • http://www.clemson.edu/assessment/assessmentpractices/referencematerials/documents/Blooms%20Taxonomy%20Action%20Verbs.pdf

  22. Example: Learning Outcomes Academic Advisors Learning Outcomes Example: • Students can independently seek out resources to aid in evaluating their progress toward degree completion • Students will be knowledgeable of the curriculum requirements for their chosen major

  23. Learning Outcome Focus 4. How does this learning outcome fit your unit? • Why did you pick this LO? Example (for fictional Office of Community Involvement): This is directly related to our unit’s purpose to promote the benefit of community involvement to our students

  24. Learning Outcome Focus 5. How does this learning outcome fit in with the YSU 2020 Strategic Plan? • Consider how the learning relates to the larger goals of the university • HLC Criterion One. Mission • The institution’s mission is clear and articulated publicly; it guides the institution’s operations. • Core Component1.A. The institution’s mission is broadly understood within the institution and guides its operations. • Sub-component 2. The institution’s academic programs, student support services, and enrollment profile are consistent with its stated mission. • HLC Criterion Three. Teaching and Learning: Quality, Resources, and Support • The institution provides high quality education, wherever and however its offerings are delivered. • Core Component 3.E.The institution fulfills the claims it makes for an enriched educational environment. • Sub-component 1. Co-curricular programs are suited to the institution’s mission and contribute to the educational experience of its students.

  25. Learning Outcome Focus Example, Questions 3-5—how do they align?

  26. Assessment and Evaluation of LO Data 6. Provide a summary of LO-related data collected for the previous academic/fiscal year (i.e., explain how you measured the LO-related activities). • Include methods used and number of individuals assessed • Minimum of one tool, but we recommend no more than two • Have one direct measure to receive an “exemplary” score • HLC Criterion Four: Teaching and Learning: Evaluation and Improvement • The institution demonstrates responsibility for the quality of its educational programs, learning environments, and support services, and it evaluates their effectiveness for student learning through processes designed to promote continuous improvement. • Core Component 4.B. The institution demonstrates a commitment to educational achievement and improvement through ongoing assessment of student learning. • Sub-component 2. The institution assesses achievement of the learning outcomes that it claims for its curricular and co-curricular programs.

  27. Examples: Direct vs. Indirect Measures Direct Measures Indirect Measures Direct evidence of learning outcome performance: • Factual knowledge questions embedded in a survey • Presentations • Performances • Simulations • Internship supervisors’ evaluations Self-reported achievement or proxy evidence: • Surveys • Group discussions • Focus groups • Exit interviews • Employment placement rates

  28. Methods: Both Direct and Indirect Tools may be direct or indirect, depending upon how they are used and their purposes:

  29. Example: Data Summary Rate of Participation (from Housing and Residence Life): Housing & Residence Life participated in the national EBI Resident Study. 618 of our 745 residents (83%) of our residents participated. This was double the rate of participation from the previous year, before “Love Where You Live” was established.

  30. Example: Data Summary Results (from Housing and Residence Life): From EBI Resident Study: • excellent scores and showed marked improvement in both factor means and national rankings. • Of 19 factors assessed in the EBI Resident Study, YSU was ranked in the top 100 of 300 participating schools in seven factors, including overall program effectiveness. • More important than rankings, our improved scores and participation reflected an increased investment in our community by our residents.

  31. Assessment and Evaluation of LO Data For questions 7-9, please provide an evaluation of the 2011-2012 data you collected related to your unit’s learning outcome. • HLC Criterion Four: Teaching and Learning: Evaluation and Improvement • The institution demonstrates responsibility for the quality of its educational programs, learning environments, and support services, and it evaluates their effectiveness for student learning through processes designed to promote continuous improvement. • Core Component 4.B. The institution demonstrates a commitment to educational achievement and improvement through ongoing assessment of student learning. • Sub-component 3. The institution uses the information gained from assessment to improve student learning.

  32. Assessment and Evaluation of LO Data 7. Based on the data, what is one key strength of (student, staff, or faculty) learning? 8. Based on the data, what is one important challenge to (student, staff, or faculty) learning? • Strengths/challenges of learning, not of your unit • You can bring in other sources of information, but be sure that the conclusions you form are based on the data you’ve collected

  33. Example: Data Evaluation Key Strength Example (from Housing): Our positive assessment results tell us that our campaign is making a difference—students are more aware of our staff’s efforts to create a more conducive living space, students are more intentional about maintaining a strong community and are more respectful of their fellow residents.

  34. Example: Data Evaluation Challenge (from Housing): • engaging our residents to remain actively involved in the residence hall and University community. • generating new ideas that keep our programs evolving. • increasing faculty awareness about our programs & incorporating more faculty into our programs.

  35. Assessment and Evaluation of LO Data 9. What action step has been identified based on this evaluation of the data? What is the status of this action step? • Considering the evaluation you conducted in questions 7 & 8, what could be changed in order to improve the learning outcome you are examining? • Data may not indicate the need for change; this is fine. Include evidence for your unit’s conclusion based on the data.

  36. Example: Action Step Action Step Example (from Housing): The Director of Housing & Residence Life conducted Speak Out Be Heard meetings in each residence hall to assess what University Housing could do to increase the desirability of student housing. Many of the comments made in those meetings became improvement projects and several have already been completed.

  37. Relation to New HLC Criteria 10. (OPTIONAL FOR 2012) Please provide an example of evidence from your unit for the new HLC Criterion Five (Resources, Planning, and Institutional Effectiveness), Core Component 5A (The institution engages in systematic and integrated planning.), Sub-component 2 (The institution links its processes for assessment of student learning, evaluation of operations, planning, and budgeting.): • Can help to begin familiarizing yourself with new HLC Criteria • How will assessment activities you are planning impact your planning and budget decisions? • How does your work help contribute to YSU’s body of evidence for this criterion?

  38. Example: HLC Criteria Aligning Assessment with Budget Example (from fictional Office of Community Engagement): While we didn’t think of it this way before reading this question, the 2010 survey results made it clear that more staff and financial resources needed to be devoted to a campus campaign. When planning for the 2011-12 fiscal year, funding was allocated specifically to the student awareness campaign, and staff responsibilities were arranged to allow for sufficient attention to outreach activities during the year.

  39. Use of Data 11. Feedback Loop: How and when are both internal and external stakeholder involved in the assessment process (e.g., data sharing and recommendation gathering)? • share with groups outside your unit • use active ways rather than passive ways to share findings • seek recommendations from groups inside and outside the unit • HLC Criterion Four: Teaching and Learning: Evaluation and Improvement • The institution demonstrates responsibility for the quality of its educational programs, learning environments, and support services, and it evaluates their effectiveness for student learning through processes designed to promote continuous improvement. • Core Component 4.B. The institution demonstrates a commitment to educational achievement and improvement through ongoing assessment of student learning. • Sub-component 4. The institution’s processes and methodologies to assess student learning reflect good practice, including the substantial participation of faculty and other instructional staff members.

  40. Example: Feedback Loop Feedback Loop Example (from fictional Office of Community Engagement): We compiled the results from our survey and shared the information by: meeting monthly with internal staff to present data and solicit recommendations; soliciting a story about the results in the Jambar for students in March; and inviting other community service-oriented campus units (e.g., University Honors, WYSU, Alumni Services) to come in person for a presentation and discussion of the data on April 17th.

  41. Use of Data 12. Considering action steps from previous years (2008-2011), what has been an impact on student learning as a result of (one of) those action steps? • You can refer to previous reports for action steps • Only one specific example is needed • In particular, we are interested in changes you made previously that have since impacted student/faculty/staff learning • HLC Criterion Four: Teaching and Learning: Evaluation and Improvement • The institution demonstrates responsibility for the quality of its educational programs, learning environments, and support services, and it evaluates their effectiveness for student learning through processes designed to promote continuous improvement. • Core Component 4.B. The institution demonstrates a commitment to educational achievement and improvement through ongoing assessment of student learning. • Sub-component 3. The institution uses the information gained from assessment to improve student learning.

  42. Example: Previous Action Step Impact Impact of previous action steps from fictional Office of Community Engagement: The results of the 2010 student survey made it apparent to us that there was a need to increase student awareness about the opportunities for and importance of community service activities as a student. We decided that for our action step we would undertake a campus campaign to increase awareness; the campaign was completed in February 2012. The results of the survey repeated in spring 2012 shows that students had an increase in the perceived value of participation—we see that as a positive impact of the campaign and a change in the students’ learning.

  43. Use of Data 13. How did your unit utilize feedback from the OOA regarding last year’s assessment report? • If this is your first time completing a plan or report, answer “first reporting year” • This is an important part of completing a full assessment cycle/spiral/loop Assessment process image credit: http://pdx.edu/studentaffairs/assessment-planning-and-practice.

  44. Example: OOA Feedback OOA Feedback Example (from Academic Advisors): Based on feedback from OOA to develop a LO that could lend itself to direct method of measure, added a LO that included: • Students will be advised in a timely manner • Student will correctly identify their advisor (is being considered for the future)

  45. Conclusion 14. Executive Summary: Summarize the key ideas from your report • one paragraph is fine • a maximum length of one half page • allows us to compile and develop themes from 40+ units

  46. Conclusion 15. (Optional) Is there anything else you would like to share regarding your assessment report and/or is there any particular area on which you would like assistance or feedback? • This is a chance to share something that didn’t “fit” with our questions but is important to your assessment “story” • Or, it is something about which you could use our help or advice

  47. Example: Additional Information • We would love some examples of rubrics that have already been developed to measure community engagement. Do you have some instruments to share with us • We would like to use the institutional account for survey monkey. How do we go about learning more about using this survey?

  48. Example: Additional Information • We need help in having a forum to share our data from our prior year’s report. Can you help us set up a workshop or forum? • We are not certain how to incorporate a direct measure with this learning outcome-do you have any ideas

  49. New Online Reporting Format Online Assessment Report Submission Form: www.jotform.us/ysuassessment/2012nareport

  50. Thank you for your participation!

More Related