Pengalaman Sebagai Penilai: Pengukuran dan CQI

1. Pengalaman Sebagai Penilai: Pengukuran dan CQI Prof. Ir. Dr. Abdul Wahab Mohammad Fakulti Kejuruteraan

2. Tujuan • Untuk memberi perspektif pengalaman sebagai seorang penilai Lembaga Jurutera Malaysia** dalam konteks (i) Pengukuran dan (ii) CQI • Menghubungkait dengan keperluan MQF dan MQA CODE OF PRACTICE INSTITUTIONAL AUDIT

3. Outcome Based Education (OBE) • “Outcome-Based Education means clearly focusing and organizing everything in an educational system around what is essential for all students to be able to do successfully at the end of their learning experiences. This means starting with a clear picture of what is important for students to be able to do, then organizing the curriculum, instruction, and assessment to make sure this learning ultimately happens” (Spady, 1994).

4. Malaysian Qualifications Framework (MQF) • A unified system and reference point for all national qualifications • Uses nationally endorsed principles, criteria, guidelines & competency standards that set the boundaries for naming,positioning and linking qualifications • Qualifications are awarded by certified providers acting under different Acts and mechanisms • Providers operate within or outside the formal education system including individually-driven life long learning efforts and verifiable workplace training and experiences.

5. EDUCATIONAL PATHWAYS Recognition of Prior Learning (RPL) POSTGRADUATE CERTIFICATE & DIPLOMA, (POSTGRADUATE PROFESSIONAL AWARDS), FELLOW PhD & HIGHER DOCTORATE OVERSEAS QUALIFICATIONS PROFESSIONAL MASTERS (4 YRS) MASTERS BY: RESEARCH COURSE WORK COMBINED GRADUATE CERTIFICATE & DIPLOMA, BACHELOR (HONS) (3-5 Yrs) ADV DIPLOMA ADV DIPLOMA SKILLS DIPLOMA VOCATIONAL DIPLOMA STPM/ STAMMATRICULATION FOUNDATION TECHNICAL & VOCATIONAL CERTIFICATE CERTIFICATE 3 (K-worker Certificate) CERTIFICATE 2 CERTIFICATE 1 OVERSEAS QUALIFICATIONS TRANSFER DEGREE 3+0, twinning, credit transfer, advance standing, external degree SPM & other approved qualifications

6. Features of the MQF • clear criteria for defining each qualification – underpin standards & facilitate mutual recognition • Provide flexible learning pathways & systematic credit transfer for progression within & across educational sectors, including RPL within/outside the country • Not a rigid system- accommodates new qualifications that arise according to needs & growth of knowledge

7. Learning Outcomes Abilities in various domains of learning that students are expected to demonstrate as evidence of competency. MQF Domain of Learning • Mastery of body of knowledge (depth, breadth and relative difficulty of specific content) • practical or psychomotor skills (range and complexity), • scientific method, critical thinking, problem solving, autonomy in decision-making • communication skills, leadership and team work • information management and life long learning skills • personal attributes, ethics, shared values and professionalism • social responsibility and accountability • Entreprenerial and managerial skills

8. Life Long Learning & Information Management Communi -cation & team skills Critical Thinking & Scientific Approach Managerial & Entrepreneu -rial Skills Learning Outcomes PSYCHOMOTOR/ PRACTICAL/ TECHNICAL SKILLS Credits Competency level SOCIAL SKILLS & RESPONSIBILITY KNOWLEDGE PROFESSIONALISM, VALUES, ATTITUDES, ETHICS

9. 3. Learning Outcomes • The LEVELS are distinguished from each other principally by learning outcomes • Other distinguishing characteristics may include: • minimum entrance requirement, • typical duration • cumulative duration in full time equivalent (part-time courses)

11. Bachelor Degree • Bachelor degree prepare students for public career, entering post-graduate programmes and research as well as highly-skills career. It also prepares an individual to take full responsibility an autonomy in making professional decision. The bachelor’s degree is awarded to an individual who are able to: • Demonstrate knowledge and understanding of fundamental principles for a specific discipline obtained from higher level text-books; • Apply knowledge and understanding using methods that demonstrate professionalism in work; • Argue and solve problems within the respective study discipline; • Use the technique to find and apply data for decision making by taking into account the relevant social, scientific and ethics issues; • Communicate and deliver effectively information, ideas, problems and solutions to expert and non-expert • demonstrate team work and suitable interpersonal skills for the job; • demonstrate self-study skills to pursue advanced life-long learning with higher autonomy

12. MISSION & GOALS VISION TEACHING & LEARNING LEARNING OUTCOMES Design of the programme content & structure,T/L methods, assessment methods graduates students Continuous Quality Management governance structure & administration of resources to design, deliver, monitor & enhance programmes RESOURCES Entry criteria, student support services, academic faculty, administrative staff, support staff, physical facilities, money, ICT, community & stakeholder engagement, research & links to education

13. Continuous Quality Improvement (CQI) “Simply put, it is the concept that there is always room for improvement. Within a company, it is the commitment to constantly improve operations, processes and activities in order to meet customer requirements in an efficient, consistent and cost effective manner” Industry Canada website at www.ic.gc.ca How do we apply CQI within our learning environment?

15. CQI • What are you currently doing? • What works well? • What does not work well? • What are you planning for the future?

16. Code of Practice for Institutional Audit (COPIA) and Code of Practice for Programme Accreditation (COPPA) Entrusted with and committed to ensuring quality in Higher Education

17. CODE OF PRACTICE: INSTITUTIONAL AUDIT • Section 1: An Overview of Quality Assurance in Higher Education • Section 2: Guidelines on Criteria and Standards for Higher Education Institution • Section 3: The Quality Audit • Section 4: Institutional Audit • Section 5: The Panel of Auditors • Section 6: Guidelines for the • Section 7: Institutional Audit Report

19. SELF-ASSESSMENT Self-assessment in an organizational setting, refers to a comprehensive, systematic and regular review of an organization's activities and results referenced against certain criteria/benchmark. The Self-Assessment process allows the organization to discern clearly its strengthsand areas in which improvements can be made and culminates in planned improvement actionswhich are then monitored for progress

20. AREAS OR ASPECTS OF STANDARDS • Vision, mission, educational goals and learning outcomes; • Curriculum design and delivery; • Assessment of students; • Student selection and support services; • Academic staff; • Educational resources; • Programme monitoring and review; • Leadership, governance and administration; • Total continual quality improvement.

22. CRITERIA • Each area is sub-divided into specific criteria, which are operationally defined and serve as performance indicators of quality. • The criteria reflect three critical factors: input, performance and quality management factors. • Each criterion is operationally defined at two levels of attainment: benchmarked and enhanced standard

23. STANDARDS • The standard defines the expected levelof attainment for each criterion and serves as a performance indicator. • Standards are specified at two levels of attainment. • The use of two levels is to acknowledge that institutions are at different stages of development and to emphasise that quality improvement is a continuous process.

24. Benchmarked Standard The standards must be met and compliance demonstrated during institutional audit. Benchmarked standards are expressed as a “must”. Enhanced Standard The standard is in accordance with the international and national consensus on good practices in higher education. Institutions should be able to demonstrate achievement of some or all of these or that initiatives to do so have or will be taken. Achievement of these standards will vary with the stage of development of the institutions, their resources and policies. Standards for enhanced standard are expressed by a “should”.

25. LEARNING OUTCOMES • The quality of the institution is ultimately assessed by the ability of its graduates to carry out their expected roles and responsibilities in society. • This requires a clear definition of the competencies that are expected to be achieved at graduation and these should reflect the level of competencies expected in the Malaysian Qualifications Framework.

26. Outcomes for Bachelor Degree • Bachelor degree prepare students for public career, entering post-graduate programmes and research as well as highly-skills career. It also prepares an individual to take full responsibility an autonomy in making professional decision. The bachelor’s degree is awarded to an individual who are able to: • Demonstrate knowledge and understanding of fundamental principles for a specific discipline obtained from higher level text-books; • Apply knowledge and understanding using methods that demonstrate professionalism in work; • Argue and solve problems within the respective study discipline; • Use the technique to find and apply data for decision making by taking into account the relevant social, scientific and ethics issues; • Communicate and deliver effectively information, ideas, problems and solutions to expert and non-expert • demonstrate team work and suitable interpersonal skills for the job; • demonstrate self-study skills to pursue advanced life-long learning with higher autonomy

27. Benchmarked standards • The institution must formulate educational goals consistent with institution’s vision, and mission. • The institution must define the specific competencies that students should demonstrate at graduation. • The competencies must include: • mastery of body of knowledge; • practical skills; • social skills and responsibility; • ethics and professionalism; • scientific method, • critical thinking and problem solving; • communication skills and team work; • information management and lifelong learning; and • entrepreneurship and management.

28. Enhanced Standard • The link between competencies expected at graduation and those required during career undertakings and/or further studies should be specified.

29. ASSESSMENT OF STUDENTS • Student assessment is an important aspect of quality assurance because assessment methods drive student learning and the outcome of assessment is used in awarding qualifications. • Hence methods of student assessment have to be clear and must support the learning outcomes.

30. Benchmarked standards • Assessment principles, methods and practices must be compatible with learning outcomes and programme content. • The process and methods of assessment must reflect the change to the programme outcomes from any review exercise. • The assessment must encourage critical thinking, problem solving and methods to enhance intellectual skills.

31. Enhanced standards • Assessments should be reviewed from time to time to achieve integrated learning

32. ASSESSMENT METHODS Benchmarked standards • The frequency and methods of student assessment including the grading criteria and all awards must be documented and communicated to students on commencement of the programme. • The assessment methods must comprise of summative and formative evaluations as well as theory and practical assessments (where relevant). • A variety of methods and tools must be used appropriately for the given learning outcomes and competencies such as communication, problem solving, teamwork and self-directed learning. • The institution must have mechanisms to ensure the validity, reliability and fairness of the assessment system.

33. Enhanced standards • Methods of assessing should include external assessors. • The institution should evaluate and document the reliability and validity of all the assessment methods used and from time to time, review and introduce new assessment methods.

34. Forms of Assessment • Formative Assessment: “The collection of data and the feedback of the results on an ongoing basis” (G. Rogers & J. Sando, 1996) • Summative Assessment: “Designed to produce information that can be used to make decisions about the overall success of the project or process.” (G. Rogers & J. Sando, 1996)

35. Effective Assessment Programs • Characteristics include: • Aligned with the institution’s mission • Has faculty ownership and responsibility • Supported institution-wide • Based on clear and measurable outcomes • Uses multiple measures • Provides useful feedback to all parties • Leads to improvement • Established process to evaluate assessment program

37. MANAGEMENT OF STUDENT ASSESSMENT • The management of the assessment system is intrinsically linked to the institution’s responsibility as a body that confers qualifications and other awards of national and international standing. • The robustness and security of the processes and procedures related to student assessment are important in inspiring confidence in the quality of the degree that is awarded by the institution.

38. Benchmarked standards • The institution must publish its grading, assessment and appeal policies; and its practices must be consistent with the policy. • The institution must have mechanisms to ensure the security of all documents and records related to assessment

39. Enhanced standards • The curriculum or programme committee should have mechanisms to review and implement new methods of assessment. • Students’ representatives, academic staff and other stakeholders should be involved in improving the system of student assessment. • Representatives of relevant stakeholders that may include staff and students should be involved in assessment review exercises.

40. PROGRAMME MONITORING AND REVIEW • Educational programmes are improved by evaluating the structure and process of education that include: • administrative structure, • leadership and governance, • the learning environment and culture of the institution; • specific curriculum components such as the syllabus, teaching methodologies and student performance as well as general learning outcomes such as career choice and performance in further training.

41. PROGRAMME MONITORING AND REVIEW • Feedback needs to be obtained from multiple sources to strengthen the evidence-based platform of educational quality in higher education. • Evidence is gathered from students’ and graduates’ feedback, performance in examinations, longitudinal study of graduate performance and perception of significant stakeholders within and outside the institution on the strengths and weaknesses of graduates and programme relevance. • The sources of information include students, graduates as well as stakeholders in the community and amongst employers, educational and government agencies, professional organisations and postgraduate educators.

42. PROGRAMME MONITORING AND REVIEW • Measures of student performance would include information on the average study duration, assessment scores, pass and failure rates at examinations, success and dropout rates, students’ and graduates’ report about their course experience, as well as time spent by students on areas of special interest. • Evaluation of student performance in examinations can reveal very useful information. If student selection has been correctly done, a high failure rate in a programme indicates something amiss in the curriculum content, teaching-learning activities or assessment system. • The programme committee needs to monitor the pass rate in each course and investigate if the rate is too high or too low.

43. MECHANISMS FOR PROGRAMME EVALUATION Benchmarked standards • The institution must establish mechanisms and resources for programme evaluation and for monitoring the implementation and student progress. • The mechanisms for programme evaluation must include the provision of benchmarked data through the use of valid and reliable methods suchas involving experts in education and external assessors. • Feedback received must be reviewed by the appropriate committee(s) and the information channelled to those who are responsible for programme development and implementation for further actions and dealt with appropriately. • Programme evaluation must address the structure and process (e.g. teaching-learning methods) of education.

44. Enhanced standards • The institution should have its own internal validation or accreditation and re-validation or re-validation processes and mechanisms for all its programmes. • The institution should proceed only after careful review by the appropriate bodies, approval by the governing board, and any necessary review and approval by the relevant authorities.

45. TEACHER AND STUDENT FEEDBACK Benchmarked standard • Teachers and students must have appropriate channels for informing issues to the authority. Effective means of both teacher and student feedback must be systematically sought. • Enhanced standards • Teachers and students should be given feedback on the actions that have been done to address the issues that they have disclosed. • Teachers and students should be actively involved in the planning of programme evaluation and its results be used for programme improvement.

46. STUDENT PERFORMANCE Benchmarked standard • Various aspects of student performance must be analysed in relation to the mission of the institution, the curriculum and the outcomes of the programmes. • Enhanced standards • Student performance and progression should be analysed taking into consideration student background and conditions as well as entrance qualifications. • Analysis of performance should be used to provide feedback to the committees responsible for student selection, curriculum planning and student counselling

47. INVOLVEMENT OF STAKEHOLDERS Benchmarked standards • Programme evaluation must involve the governance and administration of the institution, members of the institution, employers, where appropriate; representatives of the community, education and government agencies and professional organisations. • For professional programmes, the professional bodies must be involved in programme evaluation. • Enhanced standards • For non-professional programmes the relevant industry should be invited to participate in programme evaluation. • The range of stakeholders should have access to results of the course and programme evaluation, and their views on the relevance and development of the curriculum be considered.

48. TOTAL CONTINUAL QUALITY IMPROVEMENT Benchmarked standard • The institution must establish dynamic policies, procedures and mechanisms for regular reviewing and updating of its structure, functions, strategies and core activities to assure quality and must rectify documented deficiencies. • The institution must provide evidence of the review system, the result of the conducted review, the steps undertaken to implement the changes and evidence of achievement. • There must be a relation of quality assurance with the achievement of the key performance indicators.

49. Enhanced standard • The process of continual improvement should be based on prospective studies and analyses and should lead to the revisions of the policies and practices of the institution in accordance to past experience, present activities and future perspectives.

50. SYSTEM: PROGRAMME OUTCOMES Programme Outcomes Every 2 year Input from Stake holders Direct Measures Indirect Measures Benchmarking • The following changes to the PO system: • Statement has been improved from 12 to 15 to incorporate technoprenuer etc • Specific performance criteria have been identified • Rubrics have been improved and finalised • Implementation at various level • Contribution from outside the faculty as well (PPP, Kolej kediaman) CQI Who are our stakeholders?: UKM, BEM, IEM, MQA Industrial Advisory Panel Alumni, Students, Faculty Members Who, How and When? Input from Stakeholders: Annual meeting with Industrial Advisory Board, Alumni Department and Faculty review meeting Input from Industry: Specifically through industrial training visit Input from guest lecturers, industrial visit Direct Measures (Summative): Assessment of quizzes, tests and exams Use of rubric for soft skills Assessment of reports, lab sheets Indirect Measures: Exit survey, Course survey Alumni and industry survey Kajian Halatuju Kejuruteraan 2006 External Assessors Who are involved: Deanery, Quality Coordinator, Head of Department, Head of Programme, Industrial Training Unit, Individual lecturers How and When? OBE Committee, BJK, Dept and Faculty Meeting, Faculty Retreat, IAB Meeting