1 / 34

2008

KPI and Other Statistics. Program Review Tool. Sample Program. 2008. Contents. Contents. Contents. Sample Program. Key Performance Indicator Results (2005-2008) KPI Survey Statistics Student Distribution by Year in Program KPI Overall Results Student Satisfaction

tieve
Download Presentation

2008

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. KPI and Other Statistics Program Review Tool Sample Program 2008

  2. Contents Contents Contents Sample Program • Key Performance Indicator Results (2005-2008) • KPI Survey Statistics • Student Distribution by Year in Program • KPI Overall Results • Student Satisfaction • Graduation Rate • Graduate Satisfaction • Graduate Employment Rate • Employer Satisfaction • KPI Student Satisfaction Survey Results • KPI Graduate Satisfaction Survey Results • KPI Employer Satisfaction Survey Results • Course Feedback Survey Results (2004-2007) • Program Applicant & New Registrant Statistics (2004-2007) • Technical Notes Note that when current period data is not available for any of the above data sources, the respective information slides may not be included in the presentation.

  3. 2005 2006 2007 2008 Student # of Enrolled* 295 321 314 296 # of Completed Surveys 190 261 199 195 % Represented 64% 81% 63% 66% # of Students included in KPI 178 239 188 191 Graduate # of Graduates 83 79 87 107 # of Completed Surveys 62 56 64 69 % Represented 75% 71% 74% 64% # of Graduates included in KPI 55 51 62 58 Employer # of Employers 47 47 52 57 # of Completed Surveys 18 18 18 18 % Represented 38% 38% 35% 32% # of Employers included in KPI 7 7 13 13 Key Performance Indicator - Survey Overview Sample Program • Three sets of surveys: Employer, Graduate & Student. • Same set of questions are asked for each survey at all colleges and for all programs (some variations may occur). • Surveys are conducted at the same time every year. • Evaluations are provided at three levels: overall college, program at the college level, and program at the provincial level (i.e. MTCU). *# of enrolled comes from the Student Profile Data file as provided by the college

  4. 2005 2006 2007 2008 # Students % # Students % # Students % # Students % Year 1* 70 38% 82 32% 68 35% 70 36% Year 2 74 40% 128 51% 57 29% 63 32% Year 3 41 22% 43 17% 72 37% 62 32% Total 185 100% 253 100% 197 100% 195 100% Student Distribution by Year in Program Sample Program *Note a student is determined to be in a specific year of a program based on their response to Question 2 'In which semester/term are you currently enrolled' on the survey.

  5. 2005 2006 2007 2008 4 Year Average Student Satisfaction College 75% 77% 77% 78% 77% Program 79% 80% 80% 83% 82% MTCU 76% 76% 77% 74% 76% Top Large 74% 72% 76% 73% 74% Graduation Rate College 73% 73% 75% 72% 73% Program 69% 68% 70% 71% 69% MTCU 56% 59% 58% 61% 59% Top Large 61% 61% 67% 65% 63% Graduate Satisfaction College 82% 84% 84% 84% 83% Program 80% 77% 79% 82% 80% MTCU 80% 80% 77% 84% 80% Top Large 82% 78% 73% 85% 79% Graduate Employment Rate College 91% 88% 88% 90% 89% Program 84% 88% 90% 89% 87% MTCU 88% 91% 92% 95% 91% Top Large 78% 91% 90% 95% 89% Employer Satisfaction College 96% 95% 94% 91% 93% Program 98% 89% 85% 95% 93% MTCU 100% 96% 79% 97% 93% Top Large 100% 91% 75% 94% 90% KPIs - Summary Report Sample Program

  6. Program Year KPI CAPSTONE Question 14 CAPSTONE Question 26 CAPSTONE Question 44 CAPSTONE Question 45 Year 1* 84% 97% 90% 66% 75% Year 2 79% 97% 82% 55% 58% Year 3 83% 95% 95% 57% 69% Overall Program 83% 96% 89% 60% 68% Student Satisfaction by Year Sample Program *Note a student is determined to be in a specific year of a program based on their response to Question 2 'In which semester/term are you currently enrolled' on the survey. CAPSTONE Q14 'OVERALL, your program is giving you knowledge and skills that will be useful in your future career.' CAPSTONE Q26 'The OVERALL quality of the learning experiences in this program.' CAPSTONE Q44 'The OVERALL quality of the facilities/resources in the college.' CAPSTONE Q45 'The OVERALL quality of the services in the college.'

  7. College Program MTCU Top Large KPI 78% 83% 74% 73% CAPSTONE Question Q14 85% 94% 90% 93% CAPSTONE Question Q26 78% 89% 81% 85% CAPSTONE Question Q44 66% 58% 59% 52% CAPSTONE Question Q45 65% 68% 66% 62% Student Overall Satisfaction Sample Program CAPSTONE Q14 'OVERALL, your program is giving you knowledge and skills that will be useful in your future career.' CAPSTONE Q26 'The OVERALL quality of the learning experiences in this program.' CAPSTONE Q44 'The OVERALL quality of the facilities/resources in the college.' CAPSTONE Q45 'The OVERALL quality of the services in the college.' Satisfaction ratings above are based on the 'Satisfied' + 'Very Satisfied' categories.

  8. Learning Experiences (Q3-Q14) Sample Program Q3 Provides skills/abilities specific to career Q4 Includes topics relevant to future success Q5 Teachers help you understand chosen career Q6 Develops writing skills Q7 Develops speaking skills Q8 Develops ability to solve problems using math Q9 Develops ability to work with others Q10 Develops ability to solve problems Q11 Develops computer skills Q12 Opportunities to further education after graduation Q13 Provides experience useful to future outside work Q14 OVERALL, knowledge/skills useful in career

  9. Student Importance 2008 Student Satisfaction 2008 Student Satisfaction 2007 Student Satisfaction 2006 Student Satisfaction 2005 MTCU Satisfaction 2008 Top Large Satisfaction 2008 Q3 Provides you with skills & abilities specific to your chosen career 100% 94% 97% 94% 94% 93% 93% Q4 Includes topics relevant to your future success 100% 93% 94% 93% 94% 90% 92% Q5 Has teachers who help you to understand your chosen career 99% 84% 87% 83% 82% 80% 82% Q6 Develops your writing skills 73% 42% 41% 46% 51% 40% 43% Q7 Develops your speaking skills 97% 64% 63% 59% 55% 60% 63% Q8 Develops your ability to solve problems using math techniques 71% 35% 40% 34% 34% 37% 33% Q9 Develops your ability to work with others 98% 90% 91% 88% 87% 83% 87% Q10 Develops your ability to solve problems 98% 85% 86% 84% 84% 79% 82% Q11 Develops your computer skills 98% 87% 91% 90% 96% 86% 84% Q12 Provides opportunities to further education after graduation 93% 66% 63% 58% 58% 60% 62% Q13 Provides experience useful to your future life outside of work 92% 71% 78% 75% 73% 70% 71% Q14 OVERALL, program gives knowledge & skills useful for future career 100% 96% 98% 94% 94% 90% 93% Learning Experiences (Q3-Q14) Sample Program Satisfaction ratings above are based on the 'Satisfied' + 'Very Satisfied' categories.

  10. Learning Experiences - Composite Indicators Sample Program • CAREER DEVELOPMENT • Q3 Skills and abilities specific • to career • Q4 Topics relevant to future success • Q9 Ability to work with others • Q11 Develops computer skills • Q12 Opportunities to further • education after graduation • Q14 OVERALL, knowledge & skills • useful in future career • PROBLEM SOLVING SKILLS • Q8 Ability to solve problems using • math • Q10 Ability to solve problems • COMMUNICATION SKILLS • Q6 Develops writing skills • Q7 Develops speaking skills

  11. Teachers & Courses (Q15-Q26) Sample Program Q15 Teachers’ knowledge of subjects Q16 Teachers’ up-to-date/current in fields Q17 Teachers’ presentation of material Q18 Helpfulness of teacher outside class Q19 Feedback about progress Q20 Quality of classroom learning Q21 Quality of lab/shop learning Q22 Quality of other learning experiences Q23 Field placement/clinical/co-op Q24 Course materials Q25 Lab/shop facilities/equipment Q26 OVERALL quality of learning experiences

  12. Student Importance 2008 Student Satisfaction 2008 Student Satisfaction 2007 Student Satisfaction 2006 Student Satisfaction 2005 MTCU Satisfaction 2008 Top Large Satisfaction 2008 Q15 Teachers' knowledge of subjects 100% 87% 95% 92% 93% 87% 84% Q16 Teachers up-to-date/current in fields 99% 87% 93% 90% 92% 85% 85% Q17 Teachers' presentation of material 99% 70% 77% 70% 77% 68% 68% Q18 Helpfulness of teacher outside class 99% 66% 67% 63% 61% 64% 67% Q19 Feedback about progress 100% 64% 63% 63% 63% 58% 59% Q20 Quality of classroom learning 100% 76% 79% 79% 75% 68% 70% Q21 Quality of lab/shop learning 96% 63% 74% 65% 75% 63% 60% Q22 Quality of other learning experiences 92% 67% 67% 63% 70% 60% 64% Q23 Fieldplacement/clinical/co-op 97% 45% 43% 36% 52% 40% 42% Q24 Course materials 95% 57% 60% 60% 66% 52% 56% Q25 Lab/shop facilities and equipment 97% 48% 57% 55% 63% 46% 42% Q26 OVERALL quality of learning experiences 98% 88% 95% 80% 88% 81% 86% Teachers & Courses (Q15-Q26) Sample Program Satisfaction ratings above are based on the 'Satisfied' + 'Very Satisfied' categories.

  13. Facilities Sample Program Q27 Library/resource centre Q28 Open access computer labs/resources Q31 Space for individual/group study Q34 Bookstore Q35 Recreation/athletics Q43 Comfort/cleanliness/accessibility of college facilities Q44 OVERALL quality of facilities/resources

  14. Student Importance 2008 Student Usage2008 Student Satisfaction 2008 Student Satisfaction 2007 Student Satisfaction 2006 MTCU Satisfaction 2008 Top Large Satisfaction 2008 Q27 Library/resource centre 82% 85% 45% 40% 37% 54% 44% Q28 Open access computer labs/resources 64% 64% 46% 58% 57% 55% 44% Q31 Space for individual/group study 96% 89% 35% 44% 31% 43% 32% Q34 Bookstore 98% 100% 65% 67% 62% 56% 54% Q35 Recreation/athletics 62% 36% 53% 62% 59% 61% 51% Q43 Comfort/cleanliness/accessibility of college facilities 99% --- 58% 69% 58% 59% 50% Q44 OVERALL quality of facilities/resources 98% --- 59% 70% 60% 60% 53% Facilities Sample Program Satisfaction ratings above are based on the 'Satisfied' + 'Very Satisfied' categories.

  15. Services Sample Program Q29 Peer tutoring services Q30 Special skills services Q32 Counseling/native counseling/advising services Q33 Special needs/disability services Q36 Registration/records services Q37 Health services Q38 Cafeteria/food services Q39 Safety and security services Q40 Financial aid services Q41 Co-op/field placement services Q42 Employment/placement/career services Q45 OVERALL quality of services

  16. Student Importance 2008 Student Usage2008 Student Satisfaction 2008 Student Satisfaction 2007 Student Satisfaction 2006 MTCU Satisfaction 2008 Top Large Satisfaction 2008 Q29 Peer tutoring services 49% 8% 50% 56% 56% 47% 53% Q30 Special skills services 47% 9% 39% 60% 49% 41% 39% Q32 Counselling/native counselling/advising services 60% 17% 46% 65% 57% 47% 44% Q33 Special needs/disability services 58% 6% 41% 56% 61% 46% 39% Q36 Registration/records services 96% 86% 56% 56% 55% 56% 52% Q37 Health services 85% 37% 71% 64% 70% 68% 68% Q38 Cafeteria/food services 96% 95% 45% 48% 50% 46% 39% Q39 Safety and security services 96% 55% 57% 69% 63% 59% 57% Q40 Financial aid services 83% 43% 59% 63% 59% 63% 58% Q41 Co-op/field placement services 89% 31% 42% 45% 25% 44% 38% Q42 Employment/replacement/ career services 95% 28% 52% 52% 41% 40% 44% Q45 OVERALL quality of services 99% --- 69% 70% 66% 66% 61% Services Sample Program Satisfaction ratings above are based on the 'Satisfied' + 'Very Satisfied' categories.

  17. Overall Measures College Program MTCU Top Large Q20 Job relates to college program ('yes' + 'yes, partially') 68% 85% 84% 85% Q21 Skills developed during college helped you get a job ('extremely helpful' + 'helpful') 84% 90% 91% 91% Q33 Overall satisfaction with the college preparation for the type of work you were doing ('very satisfied' + 'satisfied') 80% 90% 91% 90% Q34 Satisfaction with usefulness of college education in achieving your goals after graduation ('very satisfied' + 'satisfied') 76% 81% 84% 85% Graduate Satisfaction* with Program Aspects College Program MTCU Top Large Q22. A) Course content 87% 92% 90% 90% B) Courses were up-to-date 90% 95% 93% 93% C) Overall quality of instruction 85% 95% 87% 87% D) Equipment was up-to-date 86% 90% 82% 80% E) Preparation for the job market 75% 81% 74% 74% F) Skills developed in co-op, clinical, field placement experience, and career placement services 79% 81% 67% 67% Graduate Satisfaction Sample Program *Satisfaction ratings above are based on the 'Satisfied' + 'Very Satisfied' categories.

  18. Graduate Satisfaction (Q31/32A-R) Sample Program A) Specific job-related knowledge B) Specific job-related skills C) Oral communication D) Written communications E) Comprehension F) Math skills G) Computer skills H) Critical thinking I) Problem solving J) Research & analysis K) Teamwork L) Organization & planning M) Time management N) Quality of work O) Productivity P) Creative & innovative Q) Adaptable R) Responsible

  19. Graduate Importance 2006/2007 Graduate Satisfaction 2006/2007 Graduate Satisfaction 2005/2006 Graduate Satisfaction 2004/2005 Graduate Satisfaction 2003/2004 MTCU Satisfaction 2006/2007 Top Large Satisfaction 2006/2007 A) Specific job-related knowledge 87% 88% 87% 69% 82% 87% 89% B) Specific job-related skills 90% 94% 88% 88% 86% 93% 95% C) Oral communication 98% 84% 92% 89% 84% 88% 86% D) Written communications 89% 85% 88% 84% 86% 80% 76% E) Comprehension 100% 94% 94% 84% 89% 95% 93% F) Math skills 80% 63% 72% 70% 63% 71% 64% G) Computer skills 90% 94% 100% 96% 100% 95% 95% H) Critical thinking 96% 94% 98% 91% 89% 93% 94% I) Problem solving 92% 94% 92% 82% 84% 95% 94% J) Research & analysis 76% 86% 94% 84% 77% 88% 86% K) Teamwork 90% 98% 88% 95% 96% 94% 96% L) Organization & planning 98% 89% 94% 84% 89% 92% 89% M) Time management 98% 94% 90% 87% 87% 88% 89% N) Quality of work 100% 94% 94% 91% 89% 93% 92% O) Productivity 96% 96% 94% 87% 84% 95% 93% P) Creative & innovative 85% 94% 96% 89% 95% 92% 93% Q) Adaptable 98% 87% 92% 80% 89% 86% 86% R) Responsible 100% 95% 96% 82% 89% 96% 94% Graduate Satisfaction (Q31/32A-R) Sample Program Satisfaction ratings above are based on the 'Satisfied' + 'Very Satisfied' categories.

  20. . College Program MTCU Top Large KPI - Employer Satisfaction 91% 95% 97% 94% A) Specific job-related knowledge 87% 80% 83% 79% B) Specific job-related skills 88% 82% 86% 79% C) Oral communication 94% 100% 97% 100% D) Written communications 87% 79% 86% 78% E) Comprehension 92% 93% 95% 95% F) Math skills 77% 71% 88% 78% G) Computer skills 90% 87% 95% 89% H) Critical thinking 86% 93% 86% 89% I) Problem solving 85% 87% 86% 84% J) Research & analysis 76% 69% 85% 71% K) Teamwork 95% 93% 95% 95% L) Organization & planning 86% 99% 100% 100% M) Time management 86% 100% 97% 95% N) Quality of work 92% 95% 95% 95% O) Productivity 91% 93% 92% 89% P) Creative & innovative 82% 86% 89% 83% Q) Adaptable 93% 100% 97% 94% R) Responsible 95% 100% 97% 100% Employer Satisfaction (Q72/73A-R) Sample Program Satisfaction ratings above are based on the 'Satisfied' + 'Very Satisfied' categories.

  21. Employer Satisfaction (Q72/73A-R) Sample Program A) Specific job-related knowledge B) Specific job-related skills C) Oral communication D) Written communications E) Comprehension F) Math skills G) Computer skills H) Critical thinking I) Problem solving J) Research & analysis K) Teamwork L) Organization & planning M) Time management N) Quality of work O) Productivity P) Creative & innovative Q) Adaptable R) Responsible

  22. Employer Importance 2006/2007 Employer Satisfaction 2006/2007 Employer Satisfaction 2005/2006 Employer Satisfaction 2004/2005 Employer Satisfaction 2003/2004 MTCU Satisfaction 2006/2007 Top Large Satisfaction 2006/2007 A) Specific job-related knowledge 84% 80% 86% 71% 90% 83% 79% B) Specific job-related skills 98% 80% 79% 71% 90% 86% 79% C) Oral communication 87% 100% 99% 77% 100% 97% 100% D) Written communications 93% 77% 100% 83% 80% 86% 78% E) Comprehension 100% 93% 93% 75% 90% 95% 95% F) Math skills 73% 71% 100% 44% 89% 88% 78% G) Computer skills 100% 86% 79% 100% 100% 95% 89% H) Critical thinking 87% 93% 92% 100% 90% 86% 89% I) Problem solving 87% 87% 88% 86% 90% 86% 84% J) Research & analysis 55% 69% 85% 48% 80% 85% 71% K) Teamwork 90% 93% 100% 86% 100% 95% 95% L) Organization & planning 100% 100% 85% 100% 90% 100% 100% M) Time management 93% 100% 85% 57% 78% 97% 95% N) Quality of work 100% 95% 72% 86% 100% 95% 95% O) Productivity 99% 93% 85% 71% 89% 92% 89% P) Creative & innovative 73% 86% 77% 80% 89% 89% 83% Q) Adaptable 93% 93% 85% 100% 89% 97% 94% R) Responsible 100% 100% 92% 100% 89% 97% 100% Employer Satisfaction (Q72/73A-R) Sample Program Satisfaction ratings above are based on the 'Satisfied' + 'Very Satisfied' categories.

  23. Course Feedback Survey Results - Trends Sample Program TEACHERS 1. The professor demonstrates a good knowledge of the subject area. 2. The Professor helps me to understand and apply information. 3. The Professor provides feedback that helps me to improve my performance. 4. The Professor encourages me to actively participate in this course. 5. The Professor communicates clearly. 6. The Professor relates to students in ways which promote mutual respect. 7. The Professor evaluates my performance fairly. 8. The Professor is available for consultation. COURSES 9. The learning requirements for this course are clearly stated so that I know what is expected of me. 10. This course contributes to the program in which I am enrolled. 11. This course is well organized. 12. The learning experiences are related to the learning requirements of this course. 13. The required support materials are useful. 14. The methods used to evaluate my performance are clearly outlined in writing. 15. Tests, assignments, projects, and/or other expectations relate well to the learning requirements of this course. 16. The workload in this course is acceptable. 17. Overall, please rate the quality of this course. LEARNING EXPERIENCES 18. There is adequate access to appropriate equipment in good working order for this course. 19. The physical environment in which this course is conducted supports my learning. *Note that for 2006 calendar year, the survey was not administered in Winter 2006 due to the strike

  24. Course Feedback Survey Results - Winter 2007 & Fall 2007 Sample Program 1. The professor demonstrates a good knowledge of the subject area. 2. The Professor helps me to understand and apply information. 3. The Professor provides feedback that helps me to improve my performance. 4. The Professor encourages me to actively participate in this course. 5. The Professor communicates clearly. 6. The Professor relates to students in ways which promote mutual respect. 7. The Professor evaluates my performance fairly. 8. The Professor is available for consultation. 9. The learning requirements for this course are clearly stated so that I know what is expected of me. 10. This course contributes to the program in which I am enrolled. 11. This course is well organized. 12. The learning experiences are related to the learning requirements of this course. 13. The required support materials are useful. 14. The methods used to evaluate my performance are clearly outlined in writing. 15. Tests, assignments, projects, and/or other expectations relate well to the learning requirements of this course. 16. The workload in this course is acceptable. 17. Overall, please rate the quality of this course. 18. There is adequate access to appropriate equipment in good working order for this course. 19. The physical environment in which this course is conducted supports my learning. Percentage Ratings above are based on the ‘Agree' + ‘Strongly Agree' categories.

  25. Course Feedback Survey Results - Trends Sample Program Percentage Ratings above are based on the ‘Agree' + ‘Strongly Agree' categories.

  26. Program Applicant & New Registrant Statistics Sample Program

  27. 2004 2005 2006 2007 Program Applicants &New Registrants Program Applicants 1,154 851 605 655 MTCU Applicants 2,910 2,559 1,445 1,448 Top Large Applicants 1,926 1,551 613 636 Program New Registrants 145 149 111 126 MTCU New Registrants 429 413 300 255 Top Large New Registrants 261 193 136 132 Program Applicant & New Registrant Statistics Sample Program

  28. Observations – SAMPLE ANALYSIS Sample Program • Overall KPI • The KPI rates for this program for the current survey period are: • Student KPI satisfaction, 83%, • Graduate KPI satisfaction, 82%, • Graduate KPI employment, 89%, and • Employer KPI satisfaction, 95%. • The student and graduate KPI satisfaction rates for this program are similar to the college and the previous survey period results. • Student Satisfaction Survey • In general, student satisfaction ratings for most aspects of this program were similar to or lower than the previous year. The greatest differences occurred for: • ‘quality of classroom learning’, 10% lower, and • ‘develops your ability to solve problems’, 8% lower.

  29. Observations – SAMPLE ANALYSIS Sample Program • Student Satisfaction Survey (cont’d) • Student satisfaction ratings for this program at this college tended to be lower than the MTCU and the Top Large College. The greatest differences occurred for: • ‘feedback about your progress’, 9% lower than the MTCU and 8% lower than the Top Large College, • ‘has teachers who help you understand your chosen career’, 9% lower than the MTCU and 11% lower than the Top Large College, and • develops your writing skills’, 8% lower than the Top Large College. • Analysis of learning experience composite indicators appears to indicate similar results over the past two years.

  30. Observations – SAMPLE ANALYSIS Sample Program • Student Satisfaction Survey (cont’d) • Importance ratings for learning experiences ranged from 80% to 100%. Satisfaction ratings (79% to 90%) were generally similar or lower, with relevant differences ranging from 4% to 16%. The greatest potential for improvement appears to be for: • ‘develops your speaking skills’ (100% importance, 79% satisfaction), and • ‘develops your ability to work with others’ (98% importance, 80% satisfaction). • Importance ratings, for those aspects of teachers and courses applicable to the majority of students, ranged from 82% to 99%. Corresponding satisfaction ratings (66% to 88%) were lower, with relevant differences ranging from 4% to 23%. The greatest potential for improvement appears to be for: • ‘field placement, clinical experiences, or co-op work terms’ (90% importance, 66% satisfaction, 29% ‘not applicable’), and • ‘feedback about your progress’ (94% importance, 75% satisfaction).

  31. Observations – SAMPLE ANALYSIS Sample Program • Graduate Satisfaction Survey • Overall measures of graduate satisfaction for this program were generally similar to the college but higher than MTCU results. However, the ‘overall satisfaction with the college preparation for the type of work you were doing’ was 13% higher than the college and 9% higher than the MTCU average. • Graduate satisfaction with program aspects tended to be similar to the college overall and the MTCU average, with the exception of: • ‘preparation for the job market’, 11% higher than the college, and • ‘overall quality of instruction’, 7% higher than the MTCU. • Graduate satisfaction with the various skills and abilities tended to be similar to the previous year and to the MTCU.

  32. Observations – SAMPLE ANALYSIS Sample Program • Graduate Satisfaction Survey (cont’d) • Importance ratings for the various skills and abilities ranged from 85% to 100%. Satisfaction ratings (78% to 95%) were generally similar or lower, with relevant differences ranging from 6% to 19%. The greatest potential for improvement appears to be for: • ‘specific job-related knowledge’ (100% importance, 81% satisfaction), and • ‘time management’ (93% importance, 78% satisfaction). • Employer Satisfaction Survey • Employer satisfaction with the various skills and abilities tended to be similar to the previous year and to the MTCU. • Importance ratings for the various skills and abilities ranged from 74% to 99%. Satisfaction ratings (69% to 100%) were generally similar or lower, with relevant differences ranging from 9% to 26%. The greatest potential for improvement appears to be for: • ‘oral communication’ (99% importance, 73% satisfaction), • ‘math skills’ (95% importance, 75% satisfaction), and • ‘critical thinking’ (98% importance, 80% satisfaction).

  33. Technical Notes Sample Program • Results for a specific Program at the college may be shown for each question along with the overall “College”, and one of the following benchmarks: • “MTCU” refers to the average result for all programs across the province with the same MTCU code. • “Top Large” refers to the result for this program at the Top Large college within the same MTCU code. • Provincial-level benchmarks (e.g., MTCU and Top Large benchmarks) will be derived from the current “shared” Provincial data file provided by OCAS upon the college’s request and may differ from previously published results. • The 2007/2008 KPI Graduation Rate is based on students who started one-year programs in 2005/2006, two-year programs in 2003/2004, and three-year programs in 2001/2002, and who had graduated by 2006/2007. Graduation Rate data will be derived from the current “shared” Provincial data file provided by OCAS. • The most recent cycle of Applicant and New Registrant data is for the 2007 cycle, and is referred to as 2007. This data will be derived from the current “shared” Provincial data file provided by OCAS. • Course Feedback results are derived from the survey results for each term as collected and maintained by CCI Research Inc. and are reported on for the calendar year (i.e., from winter term through to fall term) or for the current term (i.e., Winter term 2007 and Fall term 2007). Note that for the 2006 calendar year, the survey was not administered in winter 2006 due to the strike.

  34. Technical Notes Sample Program • Statistical accuracy of results is important to determine if differences between results are statistically significant. In general, results are more accurate for larger numbers of responses. Thus, college level results are more accurate than program level results, and student survey results are more accurate than graduate survey results; with employer survey results being the least accurate. In cases where the number of responses is less than 60 and/or the return rate is less than 50%, results may be statistically unreliable and therefore statistics and commentary should be interpreted with caution. • Unweighted averages are used as follows: • The “Average KPI” averages the four KPI rates, student satisfaction, graduate satisfaction, graduate employment and employer satisfaction, into a single number. • The “4 Year Average” is a simple average of the overall rates for the past four survey periods. • Learning Experiences “Composite Indicators” for ‘career development’, ‘problem solving skills’ and ‘communication skills’ are averages of applicable questions. • Gap analysis, which involves looking at the differences between importance and satisfaction ratings, is a method for prioritizing areas for improvement.

More Related