Assurance of Learning: Why, what, who, when and how?

1. Assurance of Learning:Why, what, who, when and how? Marta Colón de Toro, SPHR Assessment Coordinator College of Business Administration UPR-Mayagüez

2. Today’s Learning Goals After completing this workshop you should be able to: • Explain the importance and purpose of assessing student learning outcomes. • Recognize AACSB’s standards on assessment. • Describe the process of assessment and the challenges it represents. • Developa plan to conduct classroom assessment. • Identify and apply appropriate instruments to directly assess student learning in the classroom. • Create and maintain a course portfolio to document your classroom assessment.

3. Input Teaching-Learning Output A Transformation Process Curriculum Seminars and Competitions Internship/Coop Student Associations Extracurricular Experiences Incoming Student Completing Student • Demographics • Academics • Experience • Talents • Attitudes/Values • Knowledge • Skills • Abilities • Experience • Attitudes and Values

4. Effective curricula is… …pertinent …coherent …strategic …dynamic …assessable …intentionally designed!

5. Objective #1 • Explain the importance and purpose of assessing student learning outcomes

6. Objective #1 Have you ever asked yourself…? Are my students learning? How much are they learning? How well are they learning? What affects their learning?

7. Objective #1 Assessment …is a way of describing student learning to an identified audience for clearly- articulated reasons. …produces information useful for communication and decision-making.

8. Objective #1 Purpose of Assessment Assessment Reveals the Gap Intended Outcomes 1. Act ethically 2. Quantitative Analysis 3. Apply models 4. Decision making skills 5. Communication skills Actual Outcomes 1. 2. 3. 4. 5. ?

9. Objective #1 For example…UPR-Utuado Misión El Programa de Contabilidad de la Universidad de Puerto Rico en Utuado, reconoce que tiene ante si un reto y compromiso con la comunidad altamente industrializada y comercializada de nuestros tiempos. Su objetivo es capacitar al estudiantado con las herramientas necesarias: destrezas, conocimientos, aptitudes y actitudes que respondan a la demanda y expectativas del mundo laboral y en los principios técnicos necesarios en la Contabilidad General, en el análisis de información de transacciones financieras en forma sistemática y organizada. La finalidad de esta tecnología es desarrollar el potencial individual de estudiantes para que puedan competir en una economía global, ampliar sus oportunidades de empleo y proveer a la comunidad de personal técnico capacitado.

10. Objective #1 Another example…Arecibo MISIÓN El programa de Administración de Empresas tiene como misión lograr la formación humana, ética, intelectual y profesional del estudiante para que provea un servicio de excelencia en la administración de las empresas privadas y públicas, y en la comunidad. Además, el estudiante desarrollará las competencias necesarias para desempeñar posiciones de liderato y enfrentar retos con la disciplina y el grado de flexibilidad que exige el ambiente cambiante empresarial, y así contribuye a su éxito profesional.

11. Objective #1 Another example…Arecibo Para lograr su misión, el Departamento tiene como metas: • Familiarizar al estudiante con los conceptos, teorías, y técnicas administrativas y fiscales de las empresas privadas y públicas. • Capacitar al estudiante para recopilar datos y analizar y presentar información de su área para que se pueda utilizar en el proceso decisional. • Fomentar en el estudiante la comprensión de su responsabilidad como ciudadano en el desarrollo de la organización, la economía, la sociedad y la cultura del país. • Capacitar al estudiante con las destrezas intelectuales, técnicas y profesionales que le permitan integrarse exitosamente a la constante innovación en la empresa y la sociedad. • Desarrollar en el estudiante la agudeza de percepción para reconocer los cambios del ambiente externo e integrar las competencias que faciliten identificar mecanismos administrativos que permitan la adaptación exitosa de la empresa.

12. Objective #1 Assess Learning to: • Provide feedback and guidance to individual students. • Assist the school and faculty members to improve courses and programs. • Assure external constituents that the institution meets its academicgoals. AACSB

13. Objective #1 Levels of Assessment Institutional Learning Goals Program Learning Goals AACSB’s Focus Course Goals Today’s Focus Lesson Objectives

14. Objective #1 When to Assess • Entry • Demographics • Academic achievement • Previous Business KSAOs • Midpoint • Measure progress / added value • Exit • Measure attainment of expected learning outcomes • Follow-up • Further education • Workforce • Personal goals

15. Objective #1 Effective assessment… • Is goal directed • Ethics, writing, critical thinking, teamwork • Uses multiple measures • Tests, essays, projects, transcripts, surveys • Show student development overtime • Entry, midpoint, exit, follow-up • Uses multiple types of information • Demographics, behavioral, perceptual

16. Objective #1 Challenges of Assessment • Motivating faculty, staff, students, administration • Needs realistic goal setting • Data management • Assuring right purpose • Requires taking action • Could be costly

17. Objective #2 • Recognize AACSB’s standards on assessment

18. Objective #2 AACSB Assurance of Learning Standards • AACSB only requires assessment at the program level. • Other levels of assessment bring support • Assessments using samples are allowed. • Assessment must use direct measures of achievement • Course-embedded exercises • Course-embedded examinations • Stand-alone examinations • Indirect measures may be used to supplement direct measures of achievement.

19. Objective #2 AACSB’s Learning Expectations at the Bachelor’s Level • Educate students in a broad range of knowledge and skills as a basis for careers in business. • Build on the students’ pre-collegiate educations to prepare them to enter and sustain careers in the business world and contribute positively in the larger society. • Provide knowledge and skills for successful performance in a complex environment requiring intellectual ability to organize work, make and communicate sound decisions, and react successfully to unanticipated events. • Develop learning abilities suitable to continue higher-level intellectual development.

20. Objective #2 Standard 15 • The school uses well documented, systematicprocesses to develop, monitor, evaluate, and revise the substance and delivery of the curricula of degree programs and toassess the impact of the curricula on learning. • Curriculum management includes inputs from all appropriateconstituencies which may include faculty, staff, administrators, students, faculty from non-business disciplines, alumni, and the business community served by the school.

21. Objective #2 Standard 16 • Adapting expectations to the school’s mission and cultural circumstances, the school specifies learning goals and demonstrates achievement of learning goals for key general, management-specific, and/or appropriate discipline-specific knowledge and skills that its students achieve in each undergraduate degree program.

22. Objective #2 Standard 17 • The bachelor’s or undergraduate level degree programs must provide sufficient time, content coverage, student effort, and student-faculty interaction to assure that the learning goals are accomplished.

23. Objective #2 Program-level learning goals • General, not specific. • general knowledge and skills • management-specific learning goals • State the broad educational expectations for each degree program. • Specify the intellectual and behavioral competencies a program is intended to instill. • Clarify how we intend for graduates to be different as a result of their completion of the program. • Need to be defined operationally in order to be assessable. • Normally, four to ten learning goals will be specified for each degree program.

24. Objective #3 • Describe the process of assessment.

25. Objective #3 The Assessment Process External Reality Vision/Mission Set Goals and Design Curriculum Improve Continuously Enable Learning Assess Learning

26. The Assessment Process Objective #3 • Establish learning goals • Consult appropriate constituencies • Outcomes-based • Specify level of learning • Identify/design enabling experiences • Courses and Extracurricular activities • Select instruments and indicators of goal attainment • Assess • Interpret and report results • Identify strategies for change

27. Establish Learning goals: : “The student will be able todiscuss the importance of assessing student learning.” Identify or design enabling activities: Brief presentation on topic Brainstorming Q&A Select: Indicator: “Participant will present 2 arguments pro/con of assessment w/o consulting notes.” Instrument: Focused Listing Assess: List 2 arguments in favor and 2 arguments against doing assessment. Measure, interpret and report results _________________________ Introduce change Revise plan Objective #3 For example… Assurance of Learning Standards

28. Objective #4 • Developa plan to conduct classroom assessment.

29. Objective #4 Where to start? PLAN! Step#1 - Revisit your course learning goals (CLGs). (Also known as: learning outcomes, course objectives) • Definition “…types of executions that the students can perform after being taught in a way that they show that they have learned what was expected.” Norman Gronlund • Purpose • Express expectations of what is to be learned in a course • Provide a context for what will be learned

30. Objective #4 Learning Outcomes… • Should answer five questions: • Are they aligned with the Program Learning Goals? • What type of learning do they address? • Do they express the appropriate learning level? • Express the “ultimate goal” • Are they outcomes-based? Use action verbs!! • List what will the student will be able to doas a result of learning. • Are they student-centered? • Do they address one element of learning?

31. Ejercicio • Al finalizar el curso, los(as) estudiantes podrán: Manejar la calculadora para calcular datos específicos como la media y la desviación … Solucionar problemas estadísticos utilizando la calculadora científica.

32. Institutional Goals College Goals Program Goals Course Goals Session Objectives “…Apply scientific inquiry methods.” “…Think logically and critically.” “…basic and applied research.” “…apply data collection techniques.” “…list steps in the scientific research process.” Objective #4 Hierarchy of Learning Goals

33. Objective #4 How to align goals Complete the following for each course you teach: ADMI 3000

34. Objective #4 Program Goal Matrix Goal #2 - “The students will use the appropriate software packages and hardware as a working tool in the daily operations of a business”.

35. Objective #4 Types of Learning • Cognitive: mental skills (Knowledge) • “…enumerate three major EEO laws…” • Affective: growth in feelings or emotional areas (Attitude) • “…will value diversity…” • Psychomotor: manual or physical skills (Skills) • “… correctly use the keyboard…”

36. Objective #4 You may want to use: • Compare & contrast, critique, justify Evaluate • Adapt, combine, compare, contrast, design, generate Bloom’s Taxonomy (Cognitive Domain) Synthesize • Correlate, diagram, distinguish, outline, infer Analyze • Determine, develop, compute, utilize, conduct Apply • Classify, explain, discuss, give examples, summarize Comprehend • Define, describe, list, reproduce, enumerate Know

37. Objective #4 Dealing with “…understand”… • Always ask yourself… • How do you know the student understands? “When he/she: • Gives examples of … • Determines the “correct” method to use… • Discusses pros and cons… • Identifies elements in a given case…etc.”

38. Objective #4 Using Measurable Action Verbs analyze compute classify collaborate compare appreciate contrast define demonstrate direct derive designate discuss display evaluate know identify infer integrate interpret justify list understand organize grasp report respond solicit state synthesize name explain

39. The Student Clarify personal goals Measure success Reduce anxiety Improve studying effectiveness The Professor Design relevant classroom material Design relevant assignments Design tests/projects Objective #4 Course goals help… Improve Teaching! Improve Learning!

40. Focused on Teaching Focuses on the professor’s effort, intent or Focused on Learning Focuses on the student’s achievement Example: “Expose the student to recent research in organizational behavior”. Example: “After completing this course the student should be able to evaluate recent research in organizational behavior”. Objective #4 Right focus! The student’s learning.

41. Objective #4 Where to start? (cont.) Step #2 - Identify/design activities to enable students to achieve each learning goal: Learning Goals 1. 2. 3. 4. 5. Teaching Methodologies • Lecture • Case analysis and discussion • Oral presentations • Essays/Reports • Team Projects • Experiential Learning Extracurricular Activities • Seminars/forums • Field trips

42. Objective #4 Complete the course matrix

43. Objective #5 • Identify and apply appropriate instruments to directly assess student learning at nd classroom levels.

44. Objective #5 Where to start? (cont.) Step# 3 - Select or design classroom assessment instruments (CATs) and indicators “Classroom assessment is a simple method faculty can use to collect feedback on how well their students are learningwhat they are being taught” • Angelo, T.A (1991)

45. Objective #5 Classroom Assessment Assumptions: • Quality of student learning is directly, although not exclusively, related to the quality of teaching. • Learning is goal oriented. • Students need to receive appropriate and focused feedback early and often • Classroom Assessment provides systematic inquiry and intellectual challenge. • Classroom Assessment does not require specialized training • With Classroom Assessment efforts, faculty (and students) enhance learning and personal satisfaction.

46. Objective #5 Classroom Assessment Techniques • Learner-Centered • Teacher-Directed • Mutually Beneficial • Formative • Context-Specific • Ongoing • Rooted in Good Teaching Practice

47. Objective #5 Three step process

48. Objective #5 Classroom Assessment Techniques Direct Measures • 1 Minute paper • Muddiest Point • One-sentence summary • Direct paraphrasing • Application cards • Empty outlines • Pro-con grids • Class journals/Portfolios • What’s the principle? • Rubrics • Indirect Measures • Student evaluation of courses • Alumni Feedback • Internship/Coop

49. Objective #5 CATs • MINUTE PAPER • Ask them… • The most important thing learned • What remains unanswered • Used in lecture/discussion courses

50. Objective #5 CATs • MUDDIEST POINT • Ask them… • Identify what is least clear to you or the most confusing point • Used for large, lower level courses