Comparing Generic Student Learning Outcomes among Fresh Graduates in the Workplace

1. Comparing Generic Student Learning Outcomes among Fresh Graduates in the Workplace Focusing on Faculty of Medicine’s Graduates

2. This report includes relevant sections of a research report submitted to the Senate Committee on Teaching & Learning for all of CUHK by Hazlett, Clarke Cheng Chun Yiu, Jack Cheung Wai Hung, Gordon Kwong Kai Sun, Sunny Plus further data from a 2nd similar survey completed only by Medicine, i.e., medical alumni from its new curriculum as the surveys were done within one yr of graduation, the medical alumni were interns

3. CUHK’s Objectives Determine if recent (within 1 yr of graduation) CUHK alumni concluded that their major undergraduate programme had developed their generic skills so that they could effectively meet job requirements in their place of employment Determine if these opinions of the alumni were consistent with those of students enrolled all CUHK programmes during 1st & last yrs of the program (as assessed by the Student Engagement Questionnaire) Hong Kong employers (as assessed by the Hong Kong Education and Manpower Bureau) Faculty of Medicine’s Added Objective Determine if alumni from old and new curriculum were different in reporting how well their programme had developed these generic skills

4. Methods: Summary Summary Surveyed all alumni graduating in 2004/5 (N= 2646) and then repeated the survey on only medical graduates in 2005/06 (N=142) Focus of survey instrument: During my employment since graduation, the academic programme that I took in CUHK has enabled me to . . . e.g., . . speak more effectively in Cantonese when communicating with clients and colleagues.” Instrument and survey method pilot tested for content & face validity Instrument’s construct validity established (construct validity = 0.91) Generalizability of results estimated in initial survey (found no evidence that non respondents had any different opinions) Methodology details and relevant analyses available

7. Results

8. Response Rates Overall Response Rate (CUHK) in 1st Survey: 1356 (51%) Fac. of Medicine Response Rate in 1st Survey: 78 (54%) Fac. of Medicine Response Rate in 2nd Survey: 73 (51%)

9. Faculty of Medicine’s Within Program Comparisons With reference to CUHK’s average for all respondents (i.e., includes those who did and did not require the skill in their job)

10. CUHK & Nursing Alumni Compared Level their Programmes Developed Generic Competencies

11. CUHK & Pharmacy Alumni Compared Level their Programmes Developed Generic Competencies

12. CUHK & Medical Alumni Compared Level their Programmes Developed Generic Competencies * * statistically significant at 0.05 (2-tailed) between old & new curriculum cohorts

13. Summary Expected differences among CUHK Faculties & programmes were often found, adding evidence to the validity level of the measured outcomes These results imply that caution is needed when attempting to make any direct comparisons among Faculties or among major programmes To evaluate the relative differences, note that on a scale from 1 to 5, the absolute differences among Faculties & progammes were not large and thus few differences are educationally meaningful

14. Comparative Findings for Faculties & Programmes Variations between programmes were more pronounced than among Faculties caution should be exercised as smaller programmes with fewer respondents can more easily exhibit wider variations than large programmes with larger number of respondents

15. Conclusions • Recent alumni from 3 programs in Faculty of Medicine are quite similar to how other CUHK alumni report that their respective programmes’ have developed the graduates’ generic competencies • Medical programme’s alumni who graduated under the new curriculum reported one statistically significant improved competency (use of computers) & that the other 8 generic competencies were directionally better than parallel measures reported by alumni from the old curriculum

16. Summary Although reported results are based on a survey instrument which met acceptable standards for construct validity, there were only reasonable consistencies between CUHK’s SEQ and this Fresh Graduates Survey’s measures for the following constructs • Language proficiency • Computer literacy • Problem Solving Ability Thus, comparative alumni ratings are likely more accurately informative for these 3 above noted competencies

17. Notwithstanding the Afore Noted Cautions most constructs measured in this survey are generic to all programmes; thus, most should be included in the student learning outcomes of any major programme or Faculty these generic skills are those that both employers and the Educational Manpower Bureau regard as relevant skills for any university graduate entering the Hong Kong workforce these skills are those which CUHK has operationally defined as appropriate for its students to acquire while they pursue a chosen discipline-based or professional-based education the investigators hope that this feedback from new alumni will prove to be useful for the Faculty in its continuing efforts to enhance its respective curricula, instructional designs, learning activities and student assessments in order to achieve desired student learning outcomes