1 / 32

Conducting programme evaluationsreviews with a view to ...

jaden
Download Presentation

Conducting programme evaluationsreviews with a view to ...

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

    1. Conducting programme evaluations/reviews with a view to sustainable quality promotion Jan Botha University of Stellenbosch South Africa Keynote Address Oman Quality Network Conference Muscat, 28 October 2008

    2. 2

    3. Outline of the paper Suggestions for the academic leader QA in higher education context Misconceptions about QA in higher education context Programme evaluations and the 80/20 principle What are we learning from programme evaluations Broader perspectives Context Literature Experiences Narrower focus on a specific issue Empirical Analytical 3

    4. Suggestions for the academic leader

    5. The predicament of the academic leader I am a dean / HoD, my research …? I have become “one of them” In the middle of the sandwich I am “herding cats” I have to manage… quality From critic to custodian

    6. Academic Leadership “Learning Leaders in times of change” Geoff Scott, Hamish Coates, Michelle Anderson (May 2008)

    7. Suggestions for the Academic Leader Use quality assurance as instrument to achieve your vision and goals for your School / Department / Programme Quality assurance can be a powerful change mechanism Use a research-informed approach to quality assurance QA? Understanding / Insight ? Action

    8. QA as an Academic / Scholarly Project QA to be approached as a form of research an intellectual enterprise driven by curiosity, creativity and pragmatics underpinned by methodologically sound inquiry accompanied by experimentation and reflection informed by theorising resulting in action, application and peer reviewed publication

    9. A Principle of Design

    10. Quality Assurance in Higher Education Context

    11. “Living with ideology in the university” Ronald Barnett* The “idea of the university” as “a site of reason” Universities are beset by many ideologies ideology is “that which is opposed to truth” Competition and Entrepreneurialism Quality Managerialism Research Access, inclusivity, multiculturalism Turn the power of ideology onto itself, move from ideologies to ideaologies – recall the idea of the university

    12. The idea of the university? 12

    13. What are universities for? Universities state their vision, mission and objectives A place of quality, a place to grow Creating futures Your Knowledge Partner Innovation Generation To advance Learning and perpetuate it to Posterity Sultan Qaboos University aspires to be an academic institution of excellence recognized both nationally and internationally, a center for learning where individuals can develop their capabilities in an environment that promotes academic achievement and research excellence; and a community where the highest moral and ethical values prevail for the purpose of both self-advancement and service to the community.

    14. International snapshot International Network for Quality Assurance Agencies in Higher Education (INQAAHE)… a world-wide association of some 200 organisations Members include Commission for Academic Accreditation (CAA), UAE Netherlands Quality Agency (NQA) Malaysian Qualifications Agency (MQA) Australian Universities Quality Agency (AUQA) Comision Nacional de Evaluacion y Acreditacion Universitaria. (CONEAU), Argentina National Accreditation Agency (NAA), Russian Federation Oman Accreditation Council (OAC)

    15. South Africa Council on Higher Education (CHE) “just hang on, it will go away, as has happened with the other acronymous bodies…” Institutional Audits Programme Accreditation National Reviews of Programmes Quality Promotion and Capacity Building Three steering instruments in Higher Education

    17. Misconceptions about Quality Assurance in Higher Education

    18. Challenging the “management” of quality Is “managerialism” not contradictory to the nature of a university ? “Management” of quality used to cover for poor quality? More management of quality results in less quality… CHE publication Academic Freedom, Institutional Autonomy and Public Accountability in South African Higher Education (August 2008) (www.che.ac.za)

    19. Misunderstandings (1) Point of departure: lecturers are doing poor quality work, we need QA to ‘catch them out” I am satisfied with 95% of the work of the people in my university/faculty/school therefore QA is not necessary Just make sure that you meet the minumim standards and “keep them off your back” Aim for the minimum Comply…

    20. Misunderstandings (2) If good academics are appointed at universities, we do not need QA systems. If the wrong people are appointed, no document (however thick it is), will ensure quality. If nobody steals, we do not need laws and police. If there are thieves, no law (however thick it is), will ensure that no theft takes place. If nobody is ill, we do not need doctors. Since we know beforehand that everyone is in any case going to die some day, the whole medical profession is senseless.

    21. The case for Quality Management Quality management is a necessary condition for quality, however, quality management is not sufficient for quality

    22. 22

    27. Programme evaluations and the 80/20 principle

    28. Acknowledgements Co-researchers Professor Carools Reinecke Emeritus Vice-Chancellor, Potchefstroom University (South Africa) Mr Kamal Bhagwandas Jogibhai University of the Witwatersrand, Johannesburg (South Africa) Research Assistant Mr André Müller, University of Stellenbosch 28

    29. The Pareto Principle Vilfred Pareto (1848-1923) Wealth distribution in 19th Century England is predictably unbalanced Most income and wealth go to a minority of people Consistent mathematical relationship between proportion of people and amount of income and wealth Principle of Imbalance Zipf’s Principle of Least Effort 20-30 % of resources accounted for 70-80% activity Jurian’s Rule of the Vital Few the 80/20 Principle 29

    30. The 80/20 Principle illustrated 30

    31. What is a programme? A social intervention A set of planned, structured learning experiences leading to a qualification 31

    32. Purposes and types of programme evaluations Evaluation outcomes are used by different role players for different purposes Mouton & Babbie (The practice of social research. Oxford, 2001): judgement-orientated evaluations improvement-orientated evaluations knowledge-orientated evaluations Trow (Academic reviews and the culture of excellence. Stockholm1994): internal supportive internal evaluative external supportive external evaluative 32

    33. Unit of evaluation/review An institution as a whole An academic organizational unit within an institution department, centre, institute, bureau, school, faculty Specific activities or processes within an institution learning and teaching programmes research programmes community engagement programmes capital campaigns 33

    34. Types of programme evaluations Evaluation of professional programmes as part of the periodic programme accreditation process conducted by professional councils National reviews of programmes conducted by a national QA agency the same programme evaluated and (re)accredited simultaneously at all institutions Internal programme evaluations/reviews (for different purposes: improvement/closure/consolidation etc.) conducted by institutions professional and formative programmes undergraduate and postgraduate programmes 34

    35. Outcomes of national reviews in SA 37 MBA programmes offered by 18 public and 9 private higher education institutions evaluated during 2003 5 fully accredited, 15 conditionally accredited 19 Master of Education programmes evaluated during 2005 7 fully accredited, 2 provisional, 5 conditional, 2 postponed, 3 de-accredited 35

    36. Challenges in programme evaluation Coherence within broader internal and external QA system Formative programmes as unit of evaluation BA, BSc, BComm, B SocSc Spread over many departments in Schools and Faculties Not to conflate “process of evaluation” with “reporting the results of an evaluation process” Too much effort and resources for outcomes? 80/20 Administrative burden Evidence 36

    37. 37

    38. Expectations of Programme Evaluations Sustainable quality promotion An instrument for change Integrated with other forms of evaluation Outcomes should justify the effort for academics and evaluators and HE managers Standard methodology used self-evaluation based on criteria/standards Self-report panel visit, report improvement action, monitoring Applicable to formative programmes 38

    39. A “Pareto Approach” to Programme Evaluation Consider the ideal situation refer to criteria / standards clustered in 11 themes simplified (and “translated”) Formulate a key question (or questions) focus, contextualise (what does this mean for us) Express the evaluation in a symbol (5 point scale) discipline to make a judgement Motivate the evaluation symbol (brief narrative, with substantiating evidence) Formulate strategies (in bullet points) 39

    40. Template for the Self-evaluation Report Programme rationale Criteria The programme is consonant with the faculty’s mission, planning and resource allocation. The design maintains an appropriate balance of theoretical, practical and experiential knowledge and skills. It has sufficient disciplinary content and theoretical depth at the appropriate level.The programme offers opportunities for community interaction.The design offers learning and career pathways to students with opportunities for articulation with other programmes within and across institutions, where possible. The existing formulation of the strategic importance of the programme was re-evaluated. 1.2 Key Question 1.3 Evaluation 1.4 Motivation 1.5 Strategy   40

    41. Criteria/standards clustered in themes Programme rationale Academic integrity Student recruitment, admission and selection Staff Learning facilitation Assessment Infrastructure and academic information sources Programme co-ordination Student success and academic support for student success Service Learning and work-based learning Programme Evaluation and Development Research basis (postgraduate programmes) 41

    42. What are we learning from programme evaluations?

    43. Case study Faculty of Sciences, Stellenbosch University Bachelor of Science in Physics; Chemistry, Mathematical Sciences, Earth Science, Biodiversity, Molecular Biology, Human Life Sciences, Sport Science, Science Education Faculty of Agri-Sciences, Stellenbosch University Bachelor of Science in Agricultural Sciences in Animal Production Systems; Agricultural Economics; Wine Production Systems; Crop Production Systems; Forestry; Food Science Faculty of Humanities, University of the Witwatersrand (Johannesburg) 43

    44. Agri-Sciences 44

    45. Agri-Sciences 45

    46. Sciences 46

    47. Sciences 47

    49. Concerns, mild concerns, commendations Serious concerns Recruiting the right students, adequately prepared; widening participation (“Omanisation”), optimal number (Agricultural Sciences and Natural Sciences) Work-based learning (Agricultural Sciences and Natural Sciences) Concerns Programme rationale (Agricultural Sciences) Infrastructure, facilities, equipment (Agricultural Sciences Commendable Infrastructure, facilities, equipment (Natural Sciences) Learning facilitation (Agricultural Sciences) Assessment (Agricultural Sciences) 49

    50. Strategies for Improvement (Selected and complied from 574 strategies for improvement listed in the self-evaluation reports)

    51. 1. Programme rationale (69) To enhance interaction with stakeholders (subject specific societies, industry, extraordinary lecturers, alumni) to broaden academic and industry specific networks (through a voluntary advisory committee)  To review/restructure the focus and content of the programme annually to ensure that module outcomes strongly reflect the programme specific and generic outcomes, reviewing programme structures annually and implementing student feedback and industry input, and developing a formal system to gather feedback from graduates on their appraisal of the course, and suggestions for improvement.  To adapt and better integrate courses, share expertise/reduce duplication and use resources more cost-effectively, to reduce lecturing load by constructing generic courses across departments and including and designing modules to fill theoretical gaps and deepen theoretical knowledge or to add attractive career paths  To define service/experiential/work based learning within the programme and incorporate electives/optional subjects to prepare students better for the work environment, extending practical training/internship periods and awarding credits for lab or field work experience; To offer more opportunities for community interaction, especially for graduate students. 51

    52. 1. Programme rationale (continued)  To communicate outcomes clearly to students; to contextualize lectures and learning experiences; to communicate information about administrative and support services to students, staff and stakeholders (advertising student assistantships effectively)  To identify and support students at risk in small groups and give academic and non-academic help and to communicate the importance of class attendance; also addressing low/increased student numbers  To sharpen recruitment and marketing efforts, improving outreaches where scholars, prospective and current students can be informed about the programme and its strategic value. (Open days, information sessions...) 52

    53. 2. Academic integrity (65) To increase reseach and benchmarking opportunities with international scholars, to ensure the programme remains at the forefront of new developments, and to encourage research/teaching interaction;  To review the undergraduate programmes annually, continuousand coherently planning of courses in terms of content, level of difficulty and credit value; constantly reviewing the curriculum, learning materials, learning methods and programme outcomes; moderating it and ensuring that the feedback from external moderators is taken into account  To increase collaboration between lecturers to improve programme cohesion, filling in theoretical gaps through new and adapted courses, and to cut out duplication; also working with timetabling administrators to free up course combinations; to provide better articulation with cognate programmes, to structure the programme to expose students to the core themes, to balance practice and theory better to ensure the relevance of prescribed courses that are presented by other departments from both within, and external to, the school/faculty  To identify and remove obstacles inhibiting the use of experential learning and increase laboratory time, re-evaluating the module composition and structuring of the programme in order to make place for a longer period of internship; to evaluate the efficacy of the practical portions of the modules and investigate coherent year-long practical modules at second and third year levels;  To communicate the reason for the approach followed during the first year of study, and maintain a challenging learning environment for students despite low/increased numbers 53

    54. 2. Academic integrity (continued) To develop generic skills (writing, information and computer) and improve the teaching of practical skills through the use of continuous assessment for all modules from second year upwards, and to identify and adapt modules fairly early in the academic programme; To focus on improving student mathematic skills; To complete computer literacy and scientific communication skills modules in the first year To manage or establish new research institutes/units/centres such as an Institute for Food Technology (IFT) or Institute of Food Science and Technology (IFST) To recruit students from historically disadvantaged backgrounds To recruit more international students 54

    55. 3. Student recruitment, admission and selection (93) To involve industry stakeholders in recruitment and funding for undergraduate scholarships; To acquire additional bursaries from all sector role players; To expand facilities and staff; To increase support staff, especially in terms of popular media and marketing; To involve subject societies in recruitment actions; To increase interaction with bursary providers about the criteria for successful students and the problems that are experienced by current bursary holders; To investigate the roles professional bodies play in promoting the image of certain careers  To review admission requirements for mathematics and science related to forest science programme; To evaluate the tempo at which graduates are employed; To investigate existing recruitment actions and material in terms of success and relevance; To monitor the extended degree programme to ascertain whether it is increasing student diversity; To monitor student numbers at the beginning of the year and consider the adjustment of entrance criteria if necessary; To review policy on ranking on application form so as to admit students who desire to follow; To review academic support available to struggling students in all years  To adapt intake in terms of quantity, quality and equity of students. To align recruitment efforts of the university, faculty, programme-committee and departments; To develop and implement mechanisms, including assessment methods, to widen access, (e.g. summer school, bridging programme); To increase the diversity of the student body in terms of South African cultural groups and international students; To create a structured marketing and recruitment plan as top priority, buying in expertise if needed, consulting with SU marketing divisions, utilizing existing marketing mechanisms, negotiating for undergraduate bursaries and addressing the student diversity; To improve the administrative implementation of admission criteria; To reconsider admission requirements at first year level to curb the high failure rate, as well as at honours level (possibly); To re-assess the modules which have become “service courses”; 55

    56. 3. Recruitment, admission and selection (continued) To make the bridging programme compulsory for students with a Gr 12 mark between 50%-56%; To visit underprivileged schools in our immediate vicinity; To sponsor prizes (e.g. book prizes) for the best Biology (now Life Sciences?) student in Grade 12 at a few selected schools; To consider an orientation period including visits to departments as well as the experimental farm before registration of second-year students to allow informed choices on major subjects; To supply information on programmes at the Expo for Young Scientists and Olympiad candidates, as well as high school science teachers; To encourage third years to attend final years’ product development presentations; To focus on the recruitment of coloured students due to the demand from Agri businesses;  To implement extended degree programmes (and first year academy) to benefit students that have to overcome academic inadequacies due to historical barriers, and help students to overcome cultural and language difficultiesTo prevent over-subscription to the course; Cap student numbers (no more laboratory space available);.; To identify the optimum number of students that can be accommodated in the programme (estimated at between 40 and 50 students), keeping economic factors in mind; To market a BSc(Hons) as a career route – both commercially and in parastatal, particularly academic spheres; To pay careful attention to entrance requirements in the next few years as the new school leaving certificate is implemented. Aim to raise entrance requirements; To critically review failures and examine the reasons for these  To initiate discussions with the government on the fragmentation of higher forestry education in South Africa; To broaden the communication base with students at first- and second year level; To develop a culture of uniqueness and excellence amongst staff members and students; To raise awareness of language issues in the academic programme 56

    57. 3. Recruitment, admission and selection (continued) To make better use of recruitment opportunities (e.g. Open Day, visits to schools); To recruit black students via industry contacts; To send promotional material (pamphlets) to all Secondary schools in the Western Cape; To represent the department at open days organized by IPS; To write an article promoting Conservation Ecology and Entomology in general and our department in particular for the popular science magazine Quest; To ensure the web-site “inspires” students; To continue with current promotions activities and transparent admission policies; To have greater flexibility within the language policy; To provide admission requirements into the program (especially as of 2009) on departmental and faculty web pages and brochures.; To advertise the programme campus wide on notice board, emphasizing the financial support through industry bursaries, especially to students from disadvantaged backgrounds.; To promote the need for a Faculty Open Day with a smaller, but more specific learner group (eg. top 10 learners within a grade with mathematics as school subject or learners from strong feeder schools) so that departments can participate more effectively. Our perception is that the University Open Day fails to recruit quality motivated students for the program. To improve the distribution of marketing material; To recruit talented learners with bursaries; To contact CPS to disseminate the information on courses in the programme, job opportunities and available bursaries; To give more students access to bridging programmes; To try to attract the best Afrikaans and English students; To attract black students with the Fundza Lushaka bursary scheme 57

    58. 4. Staffing (62) To enhance opportunities for continuous professional development; To make sure all newly appointed lecturers attend the University’s induction and training courses; To ensure on-going professional development, letting staff attend various courses (e.g Web-CT training) as the need arises. To deliver graduates who are in demand in the food and cognate industries; To develop C2 and C3 staff new skills and knowledge; To enable personnel to develop academically in their fields of specialization; To encourage research activity by giving all staff members an “off semester” with no undergraduate teaching commitments and additional training in assessment methods for large classes; To maintain good research profiles and to expose students and lecturers to current research and teaching issues and trends; To quantify time usage and check against workload document to enforce contract hours; To manage time more effectively by setting fixed consultation times  To beter utilize the annual performance evaluation process, Permanent staff members to be required to compile a personal work agreement and performance contract. This assists in quality assurance of teaching and learning, research and service; To have all teaching staff, with one exception, have a PhD; To review teaching load of staff through programme evaluation; To have positive departmental evaluations; To ensure an equitable distribution of the teaching load by a yearly review; To review and evaluate the deployment of technical staff for optimum technical support  To ensure more realistic work loads for academic staff; To align specialisation and research focus of staff needs with programme. To ensure greater cooperation and less duplication 58

    59. 4. Staffing (continued) To utilize personnel seconded from industry to the departments can assist in maintaining research capacity; To investigate mechanisms by which specialised technical / practical knowledge can be imparted to students. This may be linked to a specific academic staff member, specific tasks during the internship, and/or exposure during practicals; To ensure that student assistants are well-trained  To better communicate the benefits of an excellent diversity profile and the spin-offs for the student body, who are exposed to a new experiences, approaches and ideas. To inform the faculty through the departmental evaluations that a critical mass of academic staff and sufficient administrative support are not present within all departments in the program to effectively perform and excel in all 3 areas expected namely teaching, research and community service; To actively encourage students and lecturing staff to interact about the expectations of web presence of modules, particularly in smaller senior modules  To maintain standards and ensure quality despite the increase in student numbers; To manage part-time staff carefully; To have adequate facilities and personnel to allow students to do practical work and experiments  To address the perceived insufficient levels of remuneration to attract and retain young academics, especially for young black staff. The current personnel evaluation has shortcomings and financial incentives linked to excellence are insufficient to motivate academic staff. 59

    60. 5. Learning facilitation (70) To get more clarity on the meaning of “student centered teaching” and its implications; To encourage participation by academic staff in the activities of Centre for Teaching and Learning; To utilize WebCT to effectively communicate with large groups, but not replace lecturer-student interaction or class notes; To evaluate and compare learning material with international universities; To develop a policy on student centered teaching so that independent, enthusiastic and spontaneous learning consistently takes place; To incorporate fundamental knowledge much more overtly  To act on students’ feedback and evaluation; To give a copy of each module evaluation to the programme coordinator (improve efficiency of the feedback); To request lecturers to complete module frameworks, literature bibliographies, goals, outcomes in accordance with the module framework requirements of Senate; To review the relevance of all units; To revisit the PBL process in terms of life long learning, critical thinking and professional reasoning; To review what is lectured, the links between problems and lectures and workshops and feedback; To review problems and evaluate their incremental demands over the four years; To review decision to make curriculum books only available to students in e-format  To align all study guides with the University’s policy requirements for study guides (specifically the inclusion of module-level outcomes in study guides); Investigate ways to make class experience more stimulating; To make more use of text books and journal publications, and less class notes; Powerpoint slides always to be provided to students; To employ a variety of assessment opportunities to evaluate the facilitation of learning; To use text books and scientific publications to a greater extent and in preference to class notes; To review the modules to ensure that they contain learning opportunities for the development of these skills, without unnecessary duplication; To relook at the orientation course in first year (preparation for PBL curriculum) and add a test to ensure the content is mastered and internalized 60

    61. 5. Learning facilitation (continued) To investigate the option of providing credits for laboratory or field work experience  To adequately communicate the module outcomes to the students yearly by the chairperson and via the website; To advertise student assistantships & opportunities effectively; To expose second year students to the layout and cohesion of the programme; To involve students in more personal feedback (similar to this evaluation exercise) at a module and programme level; To highlight the fact that ability to work in a team is part of the supposed programme outcomes and establish where this is in fact done;  To assign tutors dedicated to the support of disadvantaged students; To inform departments of the need to apply computer literacy skills and gradually introduce sourcing and use of scientific literature earlier in the program (3rd-year level). Basically, do not leave all tasks, seminars and research projects for the 4th year; To discuss with computer literacy convenors options to allow mathematical science students to do fewer but more relevant modules within computer literacy; To investigate the possibility of introducing opportunities for students to improve and perfect their written and verbal communication skills at early stages in their studies; To develop oral presentation skills for senior students 61

    62. 6. Assessment (58) To ensure that all staff members are aware of departmental policies and requirements for assessment and the publication and storage of marks; To use peer-reviewing within student/study groups as an effective supplementary method of assessment; To encourage continues assessment training of staff; To change fieldwork rubrics to be more user-friendly and precise (with student input); To review the number of assessment activities that contribute to the marks and activities;To review time needed to answer questions and consider the advisability of seen vs unseen questions; To analyze all exam questions according to Bloom’s taxonomy;  To align assessment practices all course frameworks, study guides with the university’s requirements; To ensure that all tests and exams are aligned with the principles of the University’s Assessment Policy; To investigate “Turn-It-In” for electronic submission of assignments; To investigate ways to reduce the significant increase in administrative load associated with internal and external moderation; To have rigorous internal moderation, and external moderation for third year and honours modules; To establish, manage and oversee an accurate and reliable administrative system to record marks onto the SIS.  To enhance strategies to eradicate plagiarism; To make assessment challenging, in particular to test problem-solving abilities; To give more smaller tests rather than only a few major tests and an exam; To assess individuals fairly in a group assignment and CTL should be approached for assistance in this regard; To pursue a more systematic manner for collecting student feedback on each module; To improve module frameworks to include assessments details (dates, type of assessment as well as expected timeframe for feedback); To use a range of assessment methods such as a seminar, laboratory, written and oral examinations; 62

    63. 6. Assessment (continued) To clearly communicate the means by which problem-solving abilities will be assessed, i.e. the quality of the questions to be expected, the amount of insight that will be required; To update the assessment dates and weights on the website; To keep lecturing staff (and yearbooks) updated with regulations regarding assessment and moderation (internal and external) at departmental level; To communicate the different assessment methods of different modules clearly to the students; To provide reasons or motivations for giving a particular mark, especially for essay-type projects and similar essay-type exam questions  To conduct individual interviews with students scoring >30% in semester test, determine reasons, plan for support; To devise an early warning system for students who are struggling (more difficult with larger classes); To give attention to entrance requirements and streaming of students to preserve standards; To handle question papers with care to avoid corruption of the assessment process;  To monitor individual student progress in terms of the First Year Academy’s mechanisms. 63

    64. 7. Infrastructure and academic information sources (55) To make sure that teaching facilities remain to up to date; To improve, maintain and replace laboratory equipment; To challenge the insufficient budgets to support the running cost of undergraduate practical training in some courses (especially those with high student numbers); To expand laboratory space, personnel; To develop IT support services to personnel  To use our excellent library services optimally, faculty librarian specialists and proper student training and regular purchases of new DVDs and material; To investigate the possibility of using the computer users area for training and tests  To re-allocate facilities to departments so that lecture theatres, laboratories and the cellar can be in the same building; To arrange for continued access to academic & scientific information (library) once graduates are working in rural areas; To maintain the effective academic support offered by the Library, SunMedia and Information Technology. To monitor access to and choice of electronic journals To identify one or more modules to be enriched in teaching of writing skills and use of library facilities and undergraduate students should be required to submit a literature review at some point in the academic programme to ensure that the desire to continue reading and learning beyond graduation is fostered. 64

    65. 7. Infrastructure and academic information sources (cont) To improve the equipment for undergraduate practicals so that each student can complete practical assignments individually; To update the computer facilities and associated software; To optimize laboratory equipment for teaching purposes (more student-proof research equipment)  To optimally utilize the class space, modern equipment, laboratory facilities, and computer and library services  To maintain laboratory equipment; To monitor the lecture hall size : amount of students  To attend to the insufficient field laboratories; To prevent overshooting capacity of a maximum of 40 students in the programme; To challenge the tariffs quoted by the General Maintenance Section of the University for services are perceived to be inflated due to a sub-contracting system and is not seen to be consistently offering value and high quality service. 65

    66. 8. Programme coordination (61) To integrate programme inputs from different stakeholders: into curriculum development; undergraduate, post-graduate, extraordinary appointments, alumni and industry; To align teaching practices with international best practrice; To state programme outcomes (generic and specific) for all lecturers and students involved in the programme through updated module frameworks  To monitor/review service levels regularly on all levels for relevancy within the programme; To evaluate and implement final year feedback; To monitor throughput of modules and support modules; To monitor student feedback earlier in the modules to report back; To consider student representation in the Programme Committee  To institute ongoing actions to improve and evaluate the efficiency and quality of the teaching programme; To align individual courses and course materials towards the main focus of the programme, especially for new members of staff; To improve communication between departments to avoid unnecessary overlap between courses and course material; 66

    67. 8. Programme coordination (contunued) To ensure better communication between lecturers in different departments about the contents of programme; To introduce programme coordinator to 1st year students & provide contact details; To ensure regular communication between teaching administration at the University, and teaching staff and students; To introduce the programme co-ordinator to first and second year students, and provide contact details in class and on the internet; To ensure that module frameworks exist for all modules, also on the website; To hold more frequent committee meetings and to create more regular opportunities for student input and participation; To explain lines of communication and authority to students  To identify and support at risk students by tracking their records and analyzing it to help the department; To explore how the information in the Student Oracle system can be utilized; To ensure that deadlines are met; To gain better understanding of the reasons why certain students do not study successfully 67

    68. 9. Student success and academic support (60)  To provide more opportunities for individual face to face contact between lecturers and students; To develop presentation skills for lecturers and technical staff; To engages with CHSE to ensure academic staff have adequate educational background and facilitation and assessment skills; To give both new and experienced staff the opportunity to attend CTLD assessment and teaching courses; To adhere to University Teaching Assessment policy; To maintain classes on/below 40 students per class  To improve the monitoring system for 1st and 2nd year students, To monitor student success more systematically; To review mentorship strategy with a view of developing a formal approach, close contact student / lecturer; To monitor continuously that the pressure to improve the pass rate of students, especially at first year level, is not carried over on to the second- and third year level to finally compromise the integrity of the whole program; To monitor the success rate in every module in the department (and faculty) where it resides.  To adapt assessment techniques to maintain good academic standards; To make the existing mentoring program more accessible to students in the programme. To evaluate and adjust the mentoring system in order to make it more effective and to ensure continuity from 1st year onwards. To look afresh at entrance requirements and options for students who struggle To investigate the possibility of continuing the mentor system for support of students at second and third year levels, in addition to the current first year support. 68

    69. 9. Student success and academic support (continued) Improve support to part time/sessional staff in their teaching role; To teach students to improve skills rather than just to supervise and critique them  To assist students to overcome the challenge posed by language. Dubble medium options for courses could be made more readily available; To remind lecturing staff that all assessments, including small tutorial tests, should be bilingual; To encourage language use accessible to all students  To alert students to the available student support services; To scrutinise students’study records in all their modules; To improve language, numerical and cognitive skills within the programme; To let students participate in the first year Learning Academy; To assist students with the challenges of the first-year of the program. The Extended degree program (also First Year Academy) is seen as an option to improve student success; To make better use of tutor demonstrators. To extend the successful tutor system to other modules if financial support can be obtained from the Faculty; To identify at risk students earlier and offer compulsory extra tutorial time; To identify good role models for students  To provide much broader access to an “extended degree”, especially within the mathematical sciences. Come to terms with the fact that a 4-year degree is the norm and use this to plan curricula. Consider a universal 4-year programme with some students emerging with an honours; others with a BSc. 69

    70. 10. Service learning and work-based learning (43) To cooperate with stakeholders to create opportunities for WBL interaction with industry for all students; To update lists of industry knowledge partners; To explore collaboration efforts with MoUs to clarify roles; To ensure that the commercial entity is suitable for receiving undergraduate students and that there is no family connection with the student  To review and improve the WBL coordination responsibility guidelines and monitor the success and/or need for WBL through departmental evaluations; To follow up past-graduates to see how feasible their recommendations are  To adapt programmes to integrate WBL modules; To use experimental farms optimally; To lengthen period of (vacation) internships; To extend mentoring system to alumni in the industry  To ensure the commercial entity is fully aware of its WBL obligations to the student before placement (e.g. internship); To require literature searches during internships; To formalize community interaction projects with SUSPI; To consider the possibility of site visits and voluntary internships   70

    71. 10. Service learning and work-based learning (cont) To communicate to all parties the outcomes of the WBL and what the student should be achieving, To ask all module coordinators to discuss work opportunities in the context of their modules;  To monitor WBL risks continuously e.g. to provide mentoring, support and networking options for supervisors (employers); To overcome logistic difficulties of WBL in certain industries, e.g. the mining and petroleum industries in the Western Cape;  To encourage students to take initiative for finding WBL placement To address any WBL misconceptions in the industry regarding the abilities of graduates and the balance of theoretical, practical and experiential teaching and training during the academic programme. To challenge lecturing staff to guard against modules being too theoretical given the context in which they are taught. To do an impact study to measure the effectiveness of the programme in preaparing community servers for practice 71

    72. 11. Programme evalution and coordination (44) To maintain regular interaction with sector stakeholders and survey employers on the impact of the programme and levels of satisfaction; To improve liaison with industry / employers; To improve industry’s understanding of the purposes and outcomes of the programme; To review the mechanism used to give stakeholders information about curriculum changes based on their input  To implement coordinated strategies to review programme content and outcomes annually; To obtain data on student recruitment generated during departmental evaluations; To ensure that all departments obtain feedback from industries; To send evaluation forms to alumni one year after graduation; To evaluate all quality assurance practices used in the department  To adapt courses to eliminate redundancy and streamline the learning process; To evaluate the relevance and efficacy of all modules, including the practical portions thereof  To initiate a process of communication and interaction with departments at this university and at other institutions to ensure that we are offering modules of the highest standards and relevance; also consider the employability of exit level students;  To encourage final year students to continue with post-graduate studies to better their qualifications and improve their employability and develop a society for past graduates 72

    73. So what are we learning from programme evaluations? Remember: Scientists, Formative, Undergraduate Commitment to maintain high academic standards Deep concern about students Quality of students (school system); Student numbers Access and Success Not clear about “student centered teaching” and its implications Need to communicate clearly to students programme and course outcomes Programme coherence and rationale Need to cooperate and communicate with lecturers in other departments (Break through academic and organisational silo’s) Improve liaison and cooperation with “the world of work” 73

    74. Reflections and conclusions Is the selectivity justified (due to 80/20 approach)? “The 20% doctor”: all students must achieve all outcomes Each programme must meet all the criteria/standards Sampling and focus justified during evaluations? How to select evidence? Just right and just enough 74

    75. Reflections and conclusions Different strategies at different levels (Programme, Department, Faculty, Institution, Higher Education System, and beyond) Encourage creativity, research, reflection Encourage debate, discussion across silo’s Closing the loop remains one of the biggest challenges Good systems, procedures Good people (at all levels) 75

More Related