Methods for Assessing Student Learning Outcomes

1. Palestine Polytechnic University Center of Excellence in Teaching and Learning Methods for Assessing Student Learning Outcomes Dr Nabil Al-Joulani Master trainer Center of Excellence in Teaching and Learning January, 15-17, 2017

2. Workshop ILO’s Upon completing the workshop, the participants will be able to : • Designing assessment tasks that are aligned with the ILO's • Design a course syllabus that includes an innovative assessment methods. • Design rubrics for assessment purposes. • Explain to a colleague the importance of implementing innovative assessment methods. • Explain to a colleague the differences between the traditional methods of assessment and the innovative ones. • Reflect on their own assessment methods.

3. Overview Day-1 15/1/2017

4. Outcomes Assessment Model and its monitoring/evaluation indirect indicators of success evaluation incoming students inputs teaching learning graduates outcomes direct indicators of success assessment

5. What is evaluation? Evaluation is the process of observing and measuring a thing for the purpose of judging it and of determining its “value,” either by comparison to similar things, or to a standard. Evaluation of teaching means passing judgment on it as part of an administrative process. (Example : Evaluation of Courses, Evaluation of Teaching) • the process of determining significance or worth, usually by careful appraisal and study. • the analysis and comparison of actual progress vs. prior plans, oriented toward improving plans for future implementation. • part of a continuing management process consisting of planning, implementation, and evaluation; ideally with each following the other in a continuous cycle until successful completion of the activity. • the process of determining the worth or value of something. This involves assigning values to the thing or person being evaluated.

6. Purposes of Evaluation • For departmental purposes (program review) • For institutional purposes (appraisal, P&T, etc.) • For external purposes (NSSE, school ranking, accreditation, etc.) • To improve student learning in the course • To identify and correct problems • To better understand how students approach learning in the course • To Improve a course • To better align the course with your learning goals • To improve quality of activities, assignments, assessments • To find out if you achieved your goals • To develop as a teacher

7. What is Assessment ? Assessment is a systematic process of looking at student achievement within and across courses by gathering, interpreting and using information about student learning for educational improvement. ( American Association of Higher Education (AAHE)) is the process of objectively understanding the state or condition of a thing, by observation and measurement. Assessment of teaching means taking a measure of its effectiveness. “Formative” assessment is measurement for the purpose of improving it. “Summative” assessment is what we normally call “evaluation.”

9. Situating Assessment: Teaching for Learning Quality Cycle 1. What learning outcomes do you want your students to achieve, (intellectual, practical, interpersonal, and personal) as a result of taking your course? 2. How will your course help your students achieve these learning outcomes? 3. How will you know if the students on your course have achieved these learning outcomes? 4. How will you know if and how your teaching has contributed to your students’ learning outcomes? Light G., Cox R. & Calkins S. (2009

10. Situating Assessment: Teaching for Learning Quality Cycle 1. What learning outcomes do you want your students to achieve, (intellectual, practical, interpersonal, and personal) as a result of taking your course? 2. How will your course help your students achieve these learning outcomes? 3. How will you know if the students on your course have achieved these learning outcomes? 4. How will you know if and how your teaching has contributed to your students’ learning outcomes? Light G., Cox R. & Calkins S. (2009

11. Single loop of quality assurance (feedback or improvement loop)

12. Teaching and learning Reflectionevaluation standards and procedures Assessment Evaluation Change Change Double loop of quality assurance (feedback or improvement) External quality reference frameworks: benchmarking, level descriptors, key competencies, etc.

13. Alignment of Assessment • Academic Program Review • College and University strategic planning • Program and University accreditations

14. Alignment of Assessment Components of your course (content, structure, teaching methods, assignments, assessment) are aligned to the intended learning outcomes > “Backwards design”

15. Assessment aligned with types of learning outcomes >>> examples Knowledge/Understanding: tests (multiple choice, short answer), reading summaries Analysis/synthesis/evaluation: case studies, book reviews, essays Subject specific skills/abilities: research projects, laboratory work, field-work, internship Transferable skills: group-work, treasure-hunt, data-collection Attitudes/values: questionnaires, portfolios, self-assessment

16. Aligning Assessment with a Learning Outcome (Biology Example)

17. Why do assessment? • Curriculum • Instructional methodology and practice • Student services Improvement : Accountability: Accreditation • State Board of Education/ Board of Regents • Public accountability • Competition for limited resources We are to demonstrate through regular and systematic assessment that students who complete their programs, no matter where or how they are offered, have achieved a specified set of learner goals established for each program.

18. How to assess? • Identify each degree and certificate program to be assessed. • Identify student learning goals and the educational criteria and experiences for each goal. • Identify appropriate assessment methods and strategies. • Collect, analyze and interpret data. • Specify program improvements.

19. When to Assess ? • Entry assessment helps determine who should be admitted and who is prepared to benefit from which programs and courses. • Midpoint assessment occurs when students reach a crucial decision point or level of attainment in their program of studies. • Exit assessment helps determine which students have attained the prerequisite knowledge, skills, and abilities associated with the program goals. • Follow-up (Post Completion) assessment helps determine the effectiveness of the educational programs in preparing students for further education, transfer, entry or reentry into the workforce, or the students’ personal goals.

20. Types of Assessment Summative and Formative Assessment Formative Assessment (Assessment for Learning) Informal: carried out frequently and is planned at the same time as teaching Provides interactive and timely feedback and response: which leads to students recognizing the (learning) gap and closing it (it is forward-looking) In addition to feedback, includes self-monitoring Fosters life-long learning: It is empirically argued that it has the greatest impact on learning and achievement Summative Assessment (Assessment of Learning) • Is carried out at intervals when achievement has to be summarized and reported • Looks at past achievements • Adds procedures or tests to existing work • Involves only grading and feedback of grades to students • Is separated from the act of teaching • “Certifies” achievement

21. Assessment of Learning Policy makers Program Planners Supervisors Assessment for Learning Students Teachers Parents Primary Users of Assessment

22. Assessment of Learning Certify competence Sort students according to achievement Assessment for Learning Help students see and hit the target Help teachers identify and respond to student needs. Typical Uses of Assessment Types

23. Assessment of Learning Follow test administration procedures Use results to help student reach GPS Assessment for Learning Inform students of targets Modify instruction Involve students in assessment Teacher’s Role

24. Assessment of Learning Study to meet standards Take the test Strive for the highest score possible Avoid failure Assessment for Learning Strive to understand the target Act on classroom assessment to improve Encourage success Student’s Role

25. Activity: Alignment of Assessment Methods Choose 1 assessment methods and align them with the teaching methods ILO and course objective. (Backward Design) Do individually, then share with group

26. What type of learning do you want to assess? "If we always do what we've always done, we will get what we've always got." Adam Urbanski

27. What to Assess? • Knowledge outcomes - core of concepts and material knowledge • Skills outcomes - what a student can do • Attitudes and values outcomes - those faculty believe to be important • Behavioral outcomes - behaviors crucial to the curriculum’s impact

28. Bloom’s Classification of Cognitive Skills • Knowledge • Comprehension • Application • Analysis • Synthesis • Evaluation

29. Assessment for Learning Vs Assessment of Learning Assessment for learning is a powerful way of raising students achievement. It is based on the principle that students will improve most if they understand the aim of their learning, where they are in relation to this aim and how they can achieve the aim ( or close the gap in their knowledge). It is not an add-on or a project; it is central to effective teaching and learning.

30. Advantages of Assessment for Learning God Assessment for Learning Makes: • An accurate assessment – knowing what are the standards are, judging students work correctively, and making accurate assessment linked to national curriculum level. • A fair assessment – knowing the methods used are valid; • A reliable assessment – ensuring that judgments are consistent and based on a range of evidence. • A useful assessment – identifying barriers to students progress and using that information to plan and success the next steps in learning. • A focused assessment – identifying areas of a student’s learning where there are block to progression, which might, for example benefit from the attention of one-to-one tuition; • For continuity of assessment, enable better transfer between years and schools.

31. Overview Day-2 16/1/2017

32. Group Question: Why do we assess students? “The quickest way to change student learning is to change the assessment system.” Elton & Laurillard, 1979

33. Purposes of Student AssessmentSummative • To pass or fail a student • To grade or rank a student • To license to proceed • To select for future courses • To license to practice • To predict success in future courses • To predict success in employment • To select for future employment • To determine if a student is learning

34. Purposes of Student AssessmentFormative • To provide feedback to students to improve their learning • To motivate students • To diagnose a student’s strengths and weaknesses • To help students develop their skills of self-assessment • To provide a profile of what a student has learnt • To determine if students are learning

35. Purposes of Student AssessmentEvaluative • To provide feedback to teachers • To improve teaching • To evaluate a course’s strengths and weaknesses • To make the course appear ‘respectable’ and credit worthy to other institutions and employers • To determine if students are learning

36. Student Approaches to Learning Entwistle, N. (2005)

37. Assessing Learning: (Verbs) Intellectual Objectives Write; state; recall; recognize; select; reproduce; measure Identify; illustrate; represent; formulate; explain; contrast Predict; select; assess; find; show; use; solve; organize; compute compare; separate; differentiate; contrast; relate; interpret; deconstruct; interrogate; summarise; argue Judge; evaluate; support; confront; critique; generalize; conclude Design, construct, plan, produce, plan, invent, discover Knowledge (As recall) Comprehension Application Analysis Evaluation Create SURFACE DEEP Bloom B. S. 1956 , Taxonomy of Educational Objectives. 2 vols. New York: Longmans Green

38. Dimensions of Assessment

39. Dimensions of Assessment Self-Referenced CriterionReferenced Based on knowledge and skills learned on course Based on self-reflection of learning achieved on the course Validity The extent to which the assessment methods reflect student learning and the learning goals of the course Summative Formative Practicality Essentially designed to sum up someone’s achievement Essentially designed for use in helping the learning process Reliability The extent to which the results of the assessment method can be trusted Norm-Referenced Peer-Referenced Based on peer appraisal of learning achieved on the course Based on comparisons with others in the group

40. Dimensions of Assessment Self-Referenced Criterion Referenced Based on knowledge and skills learned on course Based on self-reflection of learning achieved on the course Validity The extent to which the assessment methods reflect student learning and the learning goals of the course Summative Formative Practicality Essentially designed to sum up someone’s achievement Essentially designed for use in helping the learning process Reliability The extent to which the results of the assessment method can be trusted Norm-Referenced Peer-Referenced Based on peer appraisal of learning achieved on the course Based on comparisons with others in the group

41. Dimensions of Assessment Self-Referenced Criterion Referenced Based on knowledge and skills learned on course Based on self-reflection of learning achieved on the course Validity The extent to which the assessment methods reflect student learning and the learning goals of the course Summative Formative Practicality Essentially designed to sum up someone’s achievement Essentially designed for use in helping the learning process Reliability The extent to which the results of the assessment method can be trusted Norm-Referenced Peer-Referenced Based on peer appraisal of learning achieved on the course Based on comparisons with others in the group

42. Dimensions of Assessment Self-Referenced Criterion Referenced Based on knowledge and skills learned on course Based on self-reflection of learning achieved on the course Validity The extent to which the assessment methods reflect student learning and the learning goals of the course Summative Formative Practicality Essentially designed for use in helping the learning process Essentially designed to sum up someone’s achievement Reliability The extent to which the results of the assessment method can be trusted Peer-Referenced Norm-Referenced Based on peer appraisal of learning achieved on the course Based on comparisons with others in the group

43. Dimensions of Assessment Self-Referenced Criterion Referenced Based on self-reflection of learning achieved on the course Based on knowledge and skills learned on course Validity The extent to which the assessment methods reflect student learning and the learning goals of the course Summative Formative Practicality Essentially designed to sum up someone’s achievement Essentially designed for use in helping the learning process Reliability The extent to which the results of the assessment method can be trusted Norm-Referenced Peer-Referenced Based on peer appraisal of learning achieved on the course Based on comparisons with others in the group

44. Dimensions of Assessment Criterion Referenced Self-Referenced LEARNING-CENTERED Based on knowledge and skills learned on course Based on self-reflection of learning achieved on the course Validity The extent to which the assessment methods reflect student learning and the learning goals of the course Summative Formative Practicality Essentially designed to sum up someone’s achievement Essentially designed for use in helping the learning process Reliability The extent to which the results of the assessment method can be trusted Norm-Referenced Peer-Referenced Based on peer appraisal of learning achieved on the course Based on comparisons with others in the group TEACHING-CENTERED

45. Activity: Assessment Methods Choose 1or 2 assessment methods you currently use and “map” it on to these dimensions. Do individually, then share with group

46. Re-Cap of Process Step 2: Define program goals Step 3: Define student learning outcomes Step 4: Inventory existing and needed assessment methods Step 5: Identify assessment methods for each learning outcome Step 1: Define program mission

47. Criteria of Assessment Methods Sound assessments must satisfy The following quality standards Validity, Reliabilityand Fair; Clear targets; Focused purpose; Proper method; Sound sampling; Accurate assessment free of bias and distortion.

48. Checking Validity of Assessment Method

49. Assessment Methods(Some Examples) • final exam (multiple choice) • final exam (written) • quizzes • Homework • projects • reports • presentations • cold calling • observed discussion groups • career performance • standardized tests • office hours • personal response systems