Tackling the Challenges to Professional Education Project: Reforming Professional Education for a Knowledge-based Soc

1. Tackling the Challenges to Professional Education Project: Reforming Professional Education for a Knowledge-based Society Using Criterion-Referenced Assessment to promote student learning Dr. Charles C. Chan, APSS, HKPU seminar01102003/using CRA to promote student learning01102003.ppt

2. Assessing student learning outcomes • Who: teachers, instructors, peers and even students themselves can administer the assessment of learning outcomes. • For whom: students (useful feedback for further improvement: formative evaluation), instructors (formative and summative evaluation), and school or university administration (summative and quantitative evaluation). • What: subject and content specific, and/or generic transferable skills; performance at practicum. • How: subjective grading, GPA, CRA: taxonomy of educational objectives (cognitive) and behavioral competency. seminar01102003/using CRA to promote student learning01102003.ppt

3. Criterion-referenced assessment • Judges performance against a set of pre-determined criteria; • Is independent of any other student’s results; • Increase students’ intrinsic motivation and their sense of being responsible for their learning outcomes; • Encourages the cooperation between students; • Improves the effectiveness of teachers’ feedback to students. seminar01102003/using CRA to promote student learning01102003.ppt

4. Comparison between NRA and CRA seminar01102003/using CRA to promote student learning01102003.ppt

5. CRA: The SOLO taxonomy (Biggs & Collis, 1982) seminar01102003/using CRA to promote student learning01102003.ppt

6. CRA: Behavioral Competence Taxonomy seminar01102003/using CRA to promote student learning01102003.ppt

7. Project I • Subject: Social Psychology (level 3) [01-02] • Participants: 35 undergraduate students (yr1-3) • CRA used: the SOLO taxonomy • Details: Students are required to write three short papers with subject-specific scenarios and questions provided (one baseline, one mid-term and one final). The SOLO taxonomy is used to analyze students’ assignments and students are informed of their SOLO levels as feedback of their learning outcomes after each assessment. seminar01102003/using CRA to promote student learning01102003.ppt

8. Project I – Assignment sample Short Paper I (500 words). • You and one of your secondary school friends were successful to get into a degree program in the HK PolyU and the CityU / HKU respectively after one failure attempt. Right after both of you received the rejection letters, you met each other and found that the two of you had very different emotions. • Q.1 Right after that failure attempt, recall what you and your friend came up with separately to account for that rejection. • Q.2 Compare and contrast those reasons that were from you and from your friend and whether these reasons explained the different emotional states you were in. • Q.3 By using theories-in-use, demonstrate to your friend that you understand why s/he thought the way s/he did. • Q.4 Critically analyze why your friend agreed / disagreed with your perception of why s/he felt the way s/he did. seminar01102003/using CRA to promote student learning01102003.ppt

9. Project I – Record sheet of SOLO level seminar01102003/using CRA to promote student learning01102003.ppt

10. Our Project I - Results Student learning outcome: • 2.54 (SOLO 1); 3.00 (SOLO 2); 3.25 (SOLO 3) • Correlates with instructors’ judgment/subjective ratings. • Undergraduates generally produce “multistructural” level papers. • Students gradually improve during the semester. seminar01102003/using CRA to promote student learning01102003.ppt

11. Our Project I • Instructors’ comments: seminar01102003/using CRA to promote student learning01102003.ppt

12. Our Project II • Subject: Social Work Fieldwork I, II • Participants: 103 Social work undergraduates (including both bachelor and diploma students); 40 fieldwork supervisors; • CRA used: Behavioral Competence Taxonomy (self-developed, based on Hauenstein model and APSS fieldwork manual); • Details: fieldwork supervisors rated students’ performance in practicum with BCT (baseline, mid-term and final). seminar01102003/using CRA to promote student learning01102003.ppt

13. Our Project II – Record Sheet • Please identify the level of behavioral competence: • Student: _________________________ • Supervisor: _______________________ • Period under supervision: ___________________________________ • Name of the agency: _______________________________________ • Nature of the setting: _______________________________________ • Nature of the major pre-selected piece of professional practice*: _____________________________________________________________________ • Please check the following box to identify the nature of the major practice that requires the student to perform during the time period under review (Choose either one): •  1. Require execution of relatively tangible, routine and well-defined practice procedures •  2. Require re-negotiation, re-definition and problem-solving skills in the execution of practice procedures • Remark: __________________________ • * may include planning and execution of a service program for adolescents, supportive counseling for pregnant teens etc. • 1= acquisition; 2= assimilation; • 3= adaptation; 4= performance; • 5= aspiration seminar01102003/using CRA to promote student learning01102003.ppt

14. Our Project II - Results • Students improve over the semester seminar01102003/using CRA to promote student learning01102003.ppt

15. Our Project II - Results Baseline: “Student is trying to acquire the knowledge of service and the organization.” Mid-term: “Shows potentiality in integrating her experience and practice. Asks more sensible questions and more responsive in supervision.” Final: “Student tried to apply social work values and principles into practice e.g. confidentiality of cases. Student was getting more confident in working with different client group.” seminar01102003/using CRA to promote student learning01102003.ppt

16. Our Project II - Supervisors’ comments • “It takes quite some time to be familiar with the instruments.” • “BCT offers more objective assessment than the traditional impression grading system.” • “It’s good to see students improve gradually over the semester.” • “Sometimes it is difficult to decide which level students attained.” • “Some placement setting (the nature) may limit the level of attainment a student can achieve.” seminar01102003/using CRA to promote student learning01102003.ppt

17. Summary and Conclusion • CRA benefits both students and teachers by providing valuable and objective feedback for continuous improvement in learning outcomes. seminar01102003/using CRA to promote student learning01102003.ppt

18. References • Biggs, J. B. & Collis, K. F. (1982). Evaluating the quality of learning: the SOLO taxonomy (structure of the observed learning outcome). New York: Academic. • Chalmers, D. & Fuller, R. (1996). Teaching for Learning at University: Theory and Practice. London, UK: Kogan Page Ltd. • Chan, C.C., Tsui, M.S., Chan, M., & Hong, J. (2002). Applying the SOLO Taxonomy on studetns’ learning outcomesL an empirical study. Assessment & Evaluation in Higher Education, 27,6,511-528 • Chan, C. C. & Cheng, G. H.-L. (2002). Using the SOLO Taxonomy to assess students’ written work. Submitted to Teaching of Psychology. • Chan, C. C., Cheng, G. H.-L. and Chan, M. Y.-C. (2002, June). Assessing APSS Students’ Learning Outcomes and Learning Process. Teachers’ Manual version 2.1. (Available from the Network for Health and Welfare Studies, Department of Applied Social Sciences, the Hong Kong Polytechnic University) seminar01102003/using CRA to promote student learning01102003.ppt