Present Status of Evaluation of Medical Schools in Italian University

1. Project TEMPUS IV 159328 – 1 – 2009 – FR – TEMPUS - SMHES Life Long Learning Framework for Medical Teaching Staff PresentStatus of Evaluation of MedicalSchools in ItalianUniversity Prof. Stefano Elia, DepartmentofSurgery, Italy TempusCoordinator Prof. Claudio Cortese, DepartmentofInternal Medicine Prof. Antonio Volpi, Department of Internal Medicine ROME TOR VERGATA UNIVERSITY ITALY

2. The main goals of Medical Faculty are: • To improve student’s learning process • To ensure it • To evaluate it

3. It is important to realize that true learning process can only be managed by the student himself • It occurs in his mind where teachers cannot enter • Teachers can only create a stimulating environment and support mental activities necessary to facilitate the learning process • Therefore we have to switch from the historical « teacher-centered » education to the « student-centered » education

4. Any reform of medical education has to take into consideration the simultaneous reform of: • Curriculum • Teaching strategies for an effective learning • Evaluation

5. Curriculum • The Association of American Medical Colleges (AAMC) has identified 5 main roles for the future medical doctor: • Life-long learner • Clinician • Teacher/communicator • Researcher • Manager • Any type of reform of curriculum should take these roles into due consideration

6. Teaching strategies for an effective learning • Teacher’s role is to stimulate in the student’s mind: • Creativity • Problem solving ability • Mind flexibility • Criticism skillness • Opening of ideas • Analysys and synthesys of complex material

7. Teachingstrategies for an effective learning Present worldwide medical teaching tends to emphasize scientific knowledge more than encouraging scientific comprehension, clinical reasoning, practical skills, and the development of character, empathy and integrity

8. Teachingstrategies for an effective learning Medical teachers have to adapt themselves to behavioural changes in the present society focusing their action on: • More effective practical training by systematically repeated clinical tasks • Development of new evaluation methods preferably focused on competences • Improvement of research standards • Change the negative concept ofevaluation into an information step that helps students to improve their daily work

9. Evaluation • Medical teachers traditionally think that the evaluation procedure comes at the end of the learning process • This creates a gap between student’s learning and evaluation result • Evaluation is part of the teaching/learning process • Teachers have to identify and communicate end-of-course goals to the students in order for them to actively participate in their achievement • New evaluation models have to be used during learning process e.g. through progress test

10. Progress Test • Should be available along the whole educational track and used to continuously monitor: • progression of knowledge • implementation of knowledge • possible loss of previous knowledge

11. Progress Test Advantages • It avoids the student’s approach to study because « I need to take the exam » • It gives the opportunity to identify student’s strengths and weaknesses both globally and in single topics • It evaluates long term knowledge and gives confirmation of the achievement of goals • It prevents wasting of time in repeating exams

12. Progress Test Goals • To make students identify the important concepts • To motivate students in studying • To identify weak areas that need further application • To identify weakness both in the course and teacher • To determine the final score ( give or deny pass)

13. Italian System of Education High School Bachelor Degree 3 years Master of Science 2 years PhD programs One-cycle courses (5 or 6 years) Architecture/Law/Medicine

14. Italian University System First Reform: 1999 Law n.509 By-laws and rules AUTONOMY Financial Teaching Organizational

15. 1st cycle: Degree Courses • (goal: to ensure appropriate empowerment of general scientific methods and contents and acquisition of specific professional knowledge) • 2. 2nd cycle: Postgraduate Courses and 1° level Master (goal: provide an advanced level of education to perform high quality activity in a specific environment) • 3. 3rd cycle: • Postgraduate Courses and 2°nd level Master (goal: provide knowledge and capability for functions required by peculiar professional activities) • b) Ph.D. Courses • (goal: acquisition of a correct research methodology) TEACHING AUTONOMY

16. Medical Faculty is an exception since it is articulated in a unique cycle of 6 years

17. Study Classes (Law 270) Courses of the same level that share goals and qualifying educational activities University Education Credits (Crediti Formativi Universitari - CFU) One credit = 25 working hours (comprehensive of lesson, individual study, lab, verification, etc.).

18. Teaching and research autonomy <<art and sciences are free and free is their teaching »…., meaning that the teacher is independent in exercising his profession as far as programmes and methods are concerned>> Art. 33 of Italian Constitution

19. Nucleo di Valutazione di Ateneo (University Evaluation Committee) • According to present law it has to provide by 30th April each year the National Committee for Evaluation of University System (CNVSU) the following data : • Educational offer • Students’ data • Personnel’s data • Financial data • Structures • Scientific research data

20. From 2006 (law no.262) Agenzia Nazionale di Valutazione del Sistema Universitario e della Ricerca (ANVUR) National Agency for Evaluation of University System and Research

21. Quality control system Efficiency of training process is evaluated through: a) Mean teaching hours per year and average number of CFU ( CreditiFormativiUniversitari) per student b)  Enrolled/attending students c) Externalyearlyevaluation of qualitysystem by CNVSU (Comitato Nazionale per la Valutazione del Sistema Universitario) d)  Tracking of educational process by means of : withdrawal rate in the first two years, average yearly CFUs per student and yearly percentage of graduates within planned course duration. Efficacyof training process is evaluated through: a)  Level of students’ satisfaction (anonymous questionnaire) b)  Employment rate at one, two and three years from graduation.

22. BEME guidelines for medical education • Structure by problems; • Central role of clinical education; • Investigation of emerging problems, with reference and verification of scientific literature (aiming at searching for evidence proofs); • Evaluation of evidence proofs (accessibility, reliability, relevance, pertinence) ; • Analysis of transferability of evidence into an actual clinical case; • Self-evaluation in terms of self-learning of the work being done.

23. Main reasons for resistance to change • Sluggishness that pushes to defend your own action • Recognition of clinical committments as prioritary compared to teaching tasks • Non acceptance of education as a science with its principles • Ignoring fundamental elements of the educational process • Poor interest and minor care for educational activities • Complaining for lack of services and support for educational activities

24. Life-Long Learning Framework for Medical University Teaching Staff

25. Life-Long Learning Framework for Medical University Teaching Staff Goals: 1. Professional update >> all medical doctors 2. Educational pedagogic update >> only for those physicians who play an educational role Transversal tools: a. Guidelines; b. Of scientific literature (and meta-analysis); c. Network of experts. undergraduate (to include into core curriculum) postgraduate Ability to read and correctly and constantly interpret the reviews. It is a type of self-learning that has to be taught within the basic education of the medical doctor in order to build a mental dress oriented to constant education (and self-education) .

26. Life-Long Learning Framework for Medical University Teaching Staff • Pedagogics and medicine • Epistemiological specificity: • The object of the study is a subject, the human being, that can never be reduced to an object; • It is not applied in a neutral context but in a world of values. • Clinical VS Experimental Method

27. Life-Long Learning Framework for Medical University Teaching Staff Pedagogics and Education in Medical School 1 Last goals of education. It elaborates on the values included and pursued by educational action. Educational goals Pedagogics Docimology: it assesses if , how and how much the established objectives have been achieved. Means and strategies to be used to achieve educational objectives Education

28. Life-Long Learning Framework for Medical University Teaching Staff Pedagogics and Education in Medical School 2 Therefore the educational process is made of three dimensions on which to work: Planning >> pedagogics Implementing >> education Evaluating >> docimology Core curriculum Teaching function Educational management

29. Life-Long Learning Framework for Medical University Teaching Staff Undergraduate Implementation of clinical activity Implementation of humanities: phylosophy, sociology, history of medicine, psychology Core curriculum Postgraduate for medical doctors/teachers Pedagogics (education – educational technology) Docimology (experimental pedagogics)

30. Life-Long Learning Framework for Medical University Teaching Staff • Ethical aspects • Which are the reference values of the learning subject ? • Which criteria he has to use in decisional processes that involve ethical aspects ? • Which responsibilities? • How to harmonize ethics and deontology?

31. Life-Long Learning Framework for Medical University Teaching Staff • Epistemological aspects • What is the scientific culture of contemporary medicine ? • How research models and methodological presumptions are applied? • Which competences for continuous updating ? Clinical Biomedical

32. Educational Management Academic institutions devoted to education Administrative, legal, logistic ties • Plays the role of putting into relationship each other • The management acts besides institutions and teachers in macro-planning and cooperates in the evaluation of the entire course. • Some transversal work settings may be: • Study curriculum; • Courses and lessons such as: “history of medicine”; • Contextualize humanities that live within biomedical and clinical knowledge; • Recognize and implement research methodologies; • Reconstruct the formation of fudamental concepts for professional practice such as health/disease”

33. Qualification requirements for medical university teaching staff (Teaching skills)

34. Pedagogics Normative NOT Describing Science The following hypothesis is a proposal of competence NOT a picture of the existing

35. Qualification requirements for medical university teaching staff (Teaching skills)

36. Qualification requirements for medical university teaching staff (Teaching skills) • Management competences • Frontal lesson (mimics, voice management, space occupation…) • Cooperative didactics (organization of working groups for more complex goals than simple memorization such as decision making e problem solving). • Management of groups (for the analysis fof conflicts and ability to negotiate) • Simulation and role playing (active and participating didactic to elicit transformation of knowledge into competence and to create situations which are similar to professionale ones) • Technique of case-study (decision making and individual problem solving).

37. Relational competences • Physical Setting (spaces, times, disposition of participants) • Psychological Setting (space of relation: care is not given to “removed” psychological aspects but to conditions that may be necessary to achieve different specific goals. • E.g.: leadership type, definition of roles • Communication competences • Verbal/paraverbal (oral exposition: times, ways, organization, clearness) • Written (needed to transfer the experience) • Tecnifor didactic communication (e.g. ppt, audiovisuals, electronic blackboard, videorecorded simulations, up to e-learning and teleconference or online working/study groups) Communication: content and exposition Site of power exerting on a single individual concerning the management of knowledge

38. Qualification requirements for medical university teaching staff (Teaching skills) Planning competences • Evaluation competences • Times/ways of evaluation (ex ante - in itinere - ex post) • Quantitative Methodologies (test creation) • Qualitative Methodologies (creation and interpretation) • Metareflexive Competences • Observation • Analysis • Euristics

39. Definition of methodoloigcal premises • Research methodology • Logic of discovery • Tutoring • Scaffolding • Planning of significant working settings • Observation: • Of practical and tehnical procedures • Of strong communications signals • Of weak communication signals • Of communication abilities • Of relational ailities • Of team working abilities

40. Didactic evaluation: • Teachers’ performance • 2) Students’ performance • 3) Course performance The evaluation may be quantitative (tests, questionnaires, multiple choice and filling answering, standard interview) and qualitative (non standard interview, participated observation, etnographic method) Medical didactic goals have to be related to the need for care therefore the overall evaluation of a Medical School/Faculty has to include the analysis of the relationship between personnel education and long term changes produced as an answer to individual needs.

41. Qualification requirements for medical university teaching staff (Teaching skills) How to put this model into practice? • Need for an appropriate normative • Need for “education for educators” undergraduate postgraduate 3. Evaluation system