1 / 70

SOURCES OF OUTCOME DATA

SOURCES OF OUTCOME DATA. Internal measures End of course & clerkship surveys End of year surveys Faculty survey Universal Student Rating of Instruction (USRI) Certifying exam scores Canadian Graduate Questionnaire. External measures MCC

jalia
Download Presentation

SOURCES OF OUTCOME DATA

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. SOURCES OF OUTCOME DATA • Internal measures • End of course & clerkship surveys • End of year surveys • Faculty survey • Universal Student Rating of Instruction (USRI) • Certifying exam scores • Canadian Graduate Questionnaire • External measures • MCC • Resident program directors’ evaluation of graduates • NBME-Comprehensive Basic Science Exam • CaRMs • Alumni • LCME report • Alberta Universities/Colleges Graduate Employment Survey A Jones, Associate Dean – University of Calgary

  2. Overall Rating of First Year Courses Excellent V.Good Good Fair Poor

  3. Mean Scores on Certifying Evaluations (System Courses-Yr 1; SB vs CP) A Jones, Associate Dean – University of Calgary

  4. Mean Scores on Certifying Evaluations (System Courses-Yr 2; SB vs. CP) A Jones, Associate Dean – University of Calgary

  5. ALBERTA LEARNING GRADUATE EMPLOYMENTSURVEY 2004 GRADUATES FROM 2002 MEDICINE Usefulness of Your Education in Achieving: • Research Skills 80% • Working with Others 97% • A Desire to Learn More 93% • Learn Independently 97% • Awareness of Ethical Issues 97% A Jones, Associate Dean – University of Calgary

  6. ALBERTA LEARNING GRADUATE EMPLOYMENT SURVEY 2004GRADUATES FROM 2002MEDICINE • Satisfaction with the quality of teaching in your program? 100% • Satisfaction with overall quality of your educational experience 100% • University of Alberta 83% A Jones, Associate Dean – University of Calgary

  7. ALBERTA LEARNING GRADUATE EMPLOYMENT SURVEY 2004GRADUATES FROM 2002MEDICINE • I would recommend the same program of study to someone else. 100% • Satisfaction with Relevance of Courses 96% A Jones, Associate Dean – University of Calgary

  8. Overall Quality of Education at U of C by Faculty: % Satisfied or Very Satisfied Data Source: 2002 Alberta Universities/Colleges’ Graduate Employment Survey re: 2000 Grads

  9. Strongly Agree PREPAREDNESS FOR RESIDENCY Agree No Opinion Disagree Strongly Disagree 1: I am confident that I have acquired the clinical skills required to begin a residency program 2. I have the communication skills necessary to interact with patients and health professionals 3. I have basic skills in clinical decision making and the application of evidence based information to medical practice 4. I have the fundamental understanding of the issues in social sciences of medicine 5. I have the ethical and professional values that are expected of the profession 6. I have the fundamental understanding of the basic disease mechanisms, clinical presentations and principles of diagnosis and management for common conditions Data Source: Canadian Graduate Questionnaire 2005

  10. “I AM SATISFIED WITH THE QUALITY OF MY MEDICAL EDUCATION” Data Source: Canadian Graduate Questionnaire 2003, 2004 & 2005 A Jones, Associate Dean – University of Calgary

  11. A Jones, Associate Dean – University of Calgary

  12. Performance on national exams A Jones, Associate Dean – University of Calgary

  13. 720 – U of C 888- Canadian Grads/Canadian Trained A Jones, Associate Dean – University of Calgary

  14. 328 – U of C 111- Canadian Grads/Canadian Trained

  15. A Jones, Associate Dean – University of Calgary

  16. CaRMS: PERCENT OF MATCHED STUDENTS MATCHING TO FIRST CHOICE DISCIPLINE IN 1ST ITERATION CLASSES 2001- 2005 Data Source: CaRMS A Jones, Associate Dean – University of Calgary

  17. RESIDENT DIRECTORS’ ASSESSMENT OF GRADUATES (PGY1) “Overall Performance - ability to function as a resident with a full workload” Data Source: Program Directors’ Survey Class 2000 N = 50 (71%); Class 2001 N = 45 (68%) Class 2002 N = 40 (57%); Class 2003 N = 79(90%) Class 2004 N = 76 (82%) A Jones, Associate Dean – University of Calgary

  18. Resident Program Directors’ Assessmentof 2006 Graduates Data Source: Program Directors Survey A Jones, Associate Dean – University of Calgary

  19. Resident Program Directors’ Assessmentof 2005 Graduates Data Source: Program Directors Survey A Jones, Associate Dean – University of Calgary

  20. Resident Program Directors’ Assessmentof 2004 Graduates Data Source: Program Directors Survey A Jones, Associate Dean – University of Calgary

  21. UNDERGRADUATE MEDICAL EDUCATION ALUMNI SURVEYCLASSES of 1992-2002 A Jones, Associate Dean

  22. OVERALL OPINION OF THE UNDERGRADUATE MEDICAL EDUCATION PROGRAM ALUMNI SURVEY CLASSES 1992-2002 • 97% Satisfaction with the UME program at University of Calgary • 90% Felt prepared or very prepared for Post Graduate Training • 98% Would advise their child or child of a relative or friend interested in Medicine to apply to the University of Calgary A Jones, Associate Dean – University of Calgary

  23. OPNION OF ALUMNI 1992 – 2002CURRICULUM STRENGTHS AND WEAKNESSES

  24. Why Curriculum renewal is Important “A curriculum is like water. It has the tendency to seek the lowest level of energy it can reach, and without constant renewal, it will stagnate and become putrid. To avoid stagnation alone is justification for action.” Acad Medicine Sept 1998 A Jones, Associate Dean – University of Calgary

  25. Barriers to Medical School Curriculum Changes Listed by North American Academic Deans: • Already crowded curriculum • Inadequate funding • Faculty resistance • Professional ‘turf’ issues • Scheduling conflicts Graber et al. Acad Medicine 1997 A Jones, Associate Dean – University of Calgary

  26. Curriculum – A Planned Educational Experience • Define the outcome measures. • Create an evaluation system to be sure these outcomes are realized. • Develop the pathways to get to these outcomes. Allan R Jones, MD FRCPC, Associate Dean, Undergraduate Medical Education

  27. Goals for a Revised Curriculum A revised curriculum has to be consistent with available information on clinical problem solving and reflect basic principles of adult learning. A Jones, Associate Dean – University of Calgary

  28. Medical students don’t remember or can’t use the knowledge they learned in the traditional basic science courses because the knowledge is structured into mental organizations that are not useful in the clinic Barrows, 1985 A Jones, Associate Dean – University of Calgary

  29. Problem Based Learning – Benefits • Activate prior knowledge • Learn in context of clinical problem • Interest in learning stimulated • Self directed learning encouraged • Life long learning encouraged Schmidt - Norman A Jones, Associate Dean – University of Calgary

  30. Problem Based Learning Concerns • Problem solving skills are not augmented • Significant gaps in knowledge occur • Incorrect integration of basic sciences • Tendency to engage in backward reasoning Albanese; Mitchell Academic Medicine A Jones, Associate Dean – University of Calgary

  31. Clinical Reasoning • Clinical Reasoning and clinical knowledge are interdependent. • Effective problem solving requires a large store of relevant knowledge. • Clinical expertise is linked to depth and organization of clinical knowledge. A Jones, Associate Dean – University of Calgary

  32. Problem Solving Skills In Medicine Research has proven that experts in specific domains learn knowledge and problem solving skills for each problem simultaneously. That is, knowledge acquisition and clinical reasoning go hand-in-hand. Schmidt et al 1992

  33. Efforts to Help Students Improve Clinical Reasoning Education must focus on the development of adequate knowledge structures. Teaching, coaching, supervising must strongly encourage and nurture actual knowledge organization of the students.

  34. Knowledge keeps no better than fish Alfred North Whitehead 1929

  35. Clinical Reasoning and Small Group Cases It is useful to select one model of clinical reasoning and base the tutorial discussion on it. The precise model is less important than its generic use as a framework to structure the flow of discussion. It later serves as a fall-back strategy in complicated clinical situations.

  36. Structure of Medical Knowledge in Memory Categories and Prototypes Both medical textbooks and classroom teaching abound in the limitless presentation of detailed lists of disorders. More often, both fail to provide a categorization scheme that is best suited for their retrieval in a clinical problem solving situation. Bordage Med Educ 1984

  37. Types of Curricula • Disciplinary • Systems-based • Problem-oriented • Clinical Presentations based

  38. Clinical Presentation Curriculum Faculty Identify Represented by Identify Core Competencies for Clinical Presentation Curriculum Committee Clinical Presentation Develop Plans and Monitors Curriculum Enabling Basic Science Objectives Terminal Objectives Schematic Problem Solving Pathway Course Content For the Process of Together Represent Teaching Methods Clinical Reasoning Graduation Competencies Guidelines for Learning Content Evaluation

  39. Steps in Development and Dissemination of Clinical Presentation Objectives 1. Selection of clinical problem. 2. Classification system developed to help organize knowledge needed to solve the clinical problem. 3. Key Features; Discriminating features identified of prototypic prevalent disorders.

  40. Steps in Development and Dissemination of Clinical Presentation Objectives 4. Objectives and problem solving schemes developed. 5. Distribution to Faculty for balanced input from teachers generalists, specialists, and biomedical scientists. 6. Endorsements of objectives.

  41. Steps in Development and Dissemination of Clinical Presentation Objectives 7. Dissemination of objectives. 8. Encouragement of implementation of objectives in teaching, learning, clinical practice and problem solving 9. Monitor and evaluate the translation of objectives and problem solving schemes into practice.

  42. Clinical Reasoning Student Identifies Clinical Presentation Broad Classification of Problem Schematic Problem Solving Pathway Identify Causal Alternatives and Discriminating Key Factors Differential Diagnosis Diagnosis Management Plan

  43. The Scheme • Causal Categories- pre, post and renal causes of acute renal failure • Diagnoses- specific diagnoses for each causal category • Basic sciences- Integral part- Timely presentation of content

  44. “Ask any physician of 20 years standing how he has become proficient in his art and he will reply, by constant contact with disease; and he will add that the medicine he learned in schools was totally different from the medicine he learned at the bedside.” Wm. Osler 1932

  45. Bleeding Tendency/Bruising Hx PE DDx Invest. NatHx Mgmt General Objectives Thrombocytopenia Disordered Platelet Function Congenital Coagulation Disorders Acquired Coagulation Disorders Vascular Abnormalities

  46. W Surgery V Student U Radiology T Psychiatry S Physiology R Pharmacology Q Pediatrics O Pathology N Oncology M Office of Medical Education/Informatics/Culture, Health and Illness L Office of Medical Bioethics K Obstetrics & Gynecology J Neuroscience I Microbiology H Medicine G Immunology F Genetics E Family Medicine D Community Health Sciences/Nutrition/Prevention C Biochemistry B Anesthesia A Anatomy Natural History, Prognosis & Complications of Condition Prevention, Treatment & Complications of Treatment Bleeding Tendency/ Bruising Physical Examination Differential Diagnosis History a Investigation GeneralObjectivesThrombocytopeniaDisordered Platelet FunctionCongenital Coagulation DisordersAcquired Coagulation DisordersVascular Abnormalities b c d e f g 0 1 2 3 4 5 6

  47. Schematic Problem Solving Pathway Bleeding Tendency/ Bruising Clinical Presentations Platelets Coagulation Vascular Broad Classification of Problem Causal Alternatives and Discriminating Key Factors Decreased Number Abnormal Function Congenital Acquired Congenital Acquired Differential Diagnosis

  48. Pharmacology(ASA, Heparin) Histology(Bone Marrow) Genetics(Hemophilia) Basic Science Objectives for Bruising and Bleeding Anatomy(Spleen) Physiology(Hemophilia) Pathology(Vessels) Immunology(ITP, Vasculitis)

  49. Basic Science or Biomedical Knowledge in the Undergraduate Program • The purpose of basic science teaching is to provide a scientific foundation for tasks of clinical practice such as diagnosis and therapeutics. The essential challenge of balancing depth of understanding with breadth of coverage remains. • (See p. 35, Fig. 4.1)

  50. W Surgery V Student U Radiology T Psychiatry S Physiology R Pharmacology Q Pediatrics O Pathology N Oncology M Office of Medical Education/Informatics/Culture, Health and Illness L Office of Medical Bioethics K Obstetrics & Gynecology J Neuroscience I Microbiology H Medicine G Immunology F Genetics E Family Medicine D Community Health Sciences/Nutrition/Prevention C Biochemistry B Anesthesia A Anatomy Natural History, Prognosis & Complications of Condition Prevention, Treatment & Complications of Treatment Bleeding Tendency/ Bruising Physical Examination Differential Diagnosis History a Investigation GeneralObjectivesThrombocytopeniaDisordered Platelet FunctionCongenital Coagulation DisordersAcquired Coagulation DisordersVascular Abnormalities b c d e f g 0 1 2 3 4 5 6

More Related