Assessment 101

1. Assessment 101 Marta Colón de Toro, SPHR Assessment Coordinator College of Business Administration UPR-Mayagüez

2. Today’s Learning Goals After completing this workshop you should be able to: • Explain the importance and purpose of assessing outcomes, both, academic and administrative. • Recognize accreditation standards on assessment. • Describe the process of assessment and the challenges it represents. • Developa plan to conduct assessment. • Identify and apply appropriate instruments to directly assess outcomes in your area. • Identify ways to document assessment.

3. Input University Processes Output A Transformation Process Curriculum Seminars and Competitions Internship/Coop Student Associations Extracurricular Experiences Environment/Services Completing Student Incoming Student • Knowledge • Skills • Abilities • Experience • Attitudes/Values • Demographics • Academics • Experience • Talents • Attitudes/Values

4. UPR-Mayagüez Visión"Nuestro compromiso es preparar a los mejores profesionales, llevar a cabo investigación y divulgar el conocimiento en una universidad completa, ágil y eficiente". MisiónEl Recinto Universitario de Mayagüez tiene como misión dirigir sus esfuerzos para proveer a nuestra sociedad ciudadanos educados, cultos, capaces de pensar críticamente y preparados profesionalmente en los campos de la agricultura, la ingeniería, las ciencias naturales y sociales, las humanidades y la administración de empresas, para que contribuyan al desarrollo cultural, social y económico de Puerto Rico.Este proceso está dirigido a dotar a nuestros egresados de una preparación técnica y profesionalmente sólida y a desarrollar en ellos una actitud de compromiso con Puerto Rico y con nuestro Hemisferio. Nuestros egresados deben poseer, además las destrezas y los conocimientos necesarios que les permitan participar efectivamente en la búsqueda de soluciones a los problemas que nos afectan, propulsar el enriquecimiento de las artes y la cultura, la transferencia de tecnología, y defender los valores que nos caracterizan como una sociedad democrática.

5. To achieve this mission, UPRM has set forth nine primary goals (Cert. # 96-97-603): • Have students become the central figures and “reason for being”. • Become an integral institution characterized by excellence in teaching, research, and service to the community. • Encourage an entrepreneurial spirit among students • Internationalize the institution. • Strengthen research and development. • Promote complete computerization. • Evolve into an institution that is operationally agile, efficient, and auditable. • Encourage close collaboration with and provide ample service to the community. • Establish a master plan with a focus on infrastructure and physical equipment.

6. By the time of their graduation, UPRM students will be able to: • Communicate effectively. • Identify and solve problems, think critically, and synthesize knowledge appropriate to their discipline. • Apply mathematical reasoning skills, scientific inquiry methods, and tools of information technology. • Apply ethical standards. • Recognize the Puerto Rican heritage and interpret contemporary issues. • Appraise the essential values of a democratic society. • Operate in a global context, relate to a societal context, and demonstrate respect for other cultures. • Develop an appreciation for the arts and humanities. • Recognize the need to engage in life-long learning.

7. Introduction To accomplish our Vision/Mission… Everything we do should be: …pertinent …coherent …strategic …dynamic …assessable …intentionally designed!

8. Objective #1 • Explain the importance and purpose of assessment.

9. Objective #1 Assessment helps us ask ourselves.. Are we achieving our goals? Academically Support Units Are my students learning? How much are they learning? How well are they learning? What affects their learning? Are we achieving our unit’s goals? How much are we achieving? How well are we achieving them? What affects our services?

10. Objective #1 Assessment …is a way of describing what we do (student learning or support services) to an identified audience for clearly- articulated reasons. …produces information useful for communication and decision-making.

11. Objective #1 Purpose of Assessment Assessment Reveals the Gap Intended Outcomes 1. 2. 3. 4. 5. Actual Outcomes 1. 2. 3. 4. 5. ?

12. Objective #1 Assess Learning to: • Provide feedback and guidance to individuals (students/employees). • Assist the unit members to improve programs (courses and services). • Assure external constituents that the institution meets its academicgoals.

13. Objective #1 Levels of Assessment Institutional Goals Deanship Goals Unit Goals Objectives

14. Objective #1 Effective assessment… • Is directed by goals • Uses multiple measures • Show student development over time • Uses multiple types of information

15. Objective #1 Challenges of Assessment • Motivating faculty, staff, students, administration • Needs realistic goal setting • Data management • Doing it for the right purpose • Requires taking action • Could be costly

16. Objective #2 • Recognize accreditation standards on assessment

17. Objective #2 MSCHE: Standard 7Institutional Assessment The institution has developed and implemented an assessment plan and process that evaluates its overall effectiveness in: • achieving its mission and goals; • implementing planning, resource allocation, and institutional renewal processes; • using institutional resources efficiently; providing leadership and governance; • providing administrative structures and services; demonstrating institutional integrity; • and assuring that institutional processes and resources support appropriate learning and other outcomes for its students and graduates.

18. Objective #2 MSCHE: Standard 14Assessment of Student Learning • Assessment of student learning demonstrates that the institution’s students have knowledge, skills, and competencies consistent with institutional goals and that students at graduation have achieved appropriate higher education goals.

19. Other Accrediting Bodies Objective #2 • Business – AACSB • Engineering – ABET • Chemistry – • Nursing • Library • Research grants Assessment !

20. Objective #3 • Describe the process of assessment.

21. Objective #3 The Assessment Process External Reality Vision/Mission Set Goals and Design Strategies Improve Continuously Enable Achievement of Goals Assess

22. Objective #3 For example…in a classroom • Establish Learning Goals: “The student will be able todiscuss the importance of assessing student learning.” • Identify or design enabling activities: • Brief presentation on topic • Brainstorming • Q&A • Select: • Indicator: “Participant will present 2 arguments pro/con of assessment w/o consulting notes.” • Instrument: Focused Listing • Assess: • List 2 arguments in favor and 2 arguments against doing assessment. • Measure, interpret and report results (75% - listed 2/2, 15% 1/2, 10% 2/0…) • Introduce change • Revise plan (Change goal, teaching methodology, student’s preparation…) Assurance of Learning Standards

23. Objective #3 For example…in an office • Establish Goals: “Increase student participation in cultural activities.” • Identify or design enabling activities: • Promotional campaign in classrooms • Select: • Indicator: “Increase students participation in cultural activities by 20% by the end of Academic Year 2006-2007.” • Assessment Instruments • Actual attendance records. - Quantitative • Focus groups/Opinion survey - Qualitative • Assess: Measure • Interpret and report results (75% - listed 2/2, 15% 1/2, 10% 2/0…) • Introduce change • Revise plan (Change goal, teaching methodology, student’s preparation…)

24. Objective #4 • Developa plan to conduct assessment.

25. Objective #4 Where to start? PLAN! Step#1 – Establish goals • Should answer five questions: • Are they aligned with the higher-level goals? • What type of measure do they address? • Are they realistic? • Are they outcomes-based? • Are they customer (student)-centered? Step #2 - Identify/design activities to enable achievement of each goal. Step #3 - Select or design assessment instruments and indicators. Step #4 – Assess Step #5 – Interpret and report results Step #6 – Identify and introduce change strategies Step #7 – Revise plan

26. For example: In class Objective #4 Learning Goals 1. Leadership 2. Critical Analysis 3. Interpersonal skills 4. Team Work 5. Problem Solving Teaching Methodologies • Lecture • Case analysis and discussion • Oral presentations • Essays/Reports • Team Projects • Experiential Learning • Simulations Extracurricular Activities • Seminars/forums • Field trips

27. Objective #4 For Example: In an office Office Goals 1. Increase qty/qlty 2. Reduce errors 3. Increase participation 4. Reduce time 5. Customer service Strategies Train employees Increase/try new communication channels Improve technology Re-engineer processes Relocate personnel

28. Objective #5 • Identify and apply appropriate instruments to directly assess outcomes.

29. Objective #5 Assessment Assumptions: • Quality of outcomes is directly, although not exclusively, related to the quality of inputs. • Human behavior is goal oriented. • Feedback should be received early and often. • Assessment provides systematic inquiry and challenges status quo. • Assessment does not require specialized training • Assessment efforts enhances performance and personal satisfaction.

30. Objective #5 Assessment Techniques • Learner/Customer-Centered • Teacher/Management-Directed • Mutually Beneficial • Formative • Context-Specific • Ongoing • Rooted in Good Teaching/Administration Practice

31. Objective #5 Classroom Assessment Techniques Direct Measures • 1 Minute paper • Muddiest Point • One-sentence summary • Direct paraphrasing • Application cards • Empty outlines • Pro-con grids • Class journals/Portfolios • What’s the principle? • Rubrics • Indirect Measures • Student evaluation of courses • Alumni Feedback • Internship/Coop

32. Objective #5 Administrative Assessment • Opinion/Satisfaction surveys • Daily logs • Attendance records • Quantity/quality measures • Process analysis • Workflow analysis

33. REMEMBER… • These ideas are suggestions to be adapted, not models to be adopted • Don’t ask if you don’t really want to know or are not going to do anything about it. • Start with something simple and then add up.

34. Objective #6 • Identify ways to document assessment.

35. Documenting CATs CAT’s • Diagnostic Tests • Pre-Post Test • Muddiest Point • Minute Paper • Application Cards • Direct Paraphrasing • Course Final General Assessment Documentation • Brief Description or Sample copy • Sample of Grades (H,M,L) • Sample Answers • Analysis of Outcomes • Conclusions • Recommendations • Actions Taken

36. Documenting administrative assessment • Survey samples • Descriptive statistics • Institutionalize it through • Publications • Requiring it for decision making • Create position • Assign resources

37. Analysis of Outcomes • Quantitative • Descriptive statistics • Average, Median, Mode, Standard deviation • Question by question analysis • Qualitative • Look for • Misconceptions • Attitudes/prejudices

38. Conclusions/Recommendations • Summarize lessons learned • List actions to be taken for improvement • TAKE ACTION!!! • Refer pertinent recommendations to appropriate forum • Implement changes • Assess again, and again, and again…

39. Stages of Grieving • Denial • Anger • Bargaining • Depression • Acceptance On Death and Dying by Elisabeth Kubler-Ross

40. Good luck!Keep learning!colon-m@rigel.uprm.edu787-265-3801