1 / 26

Fundamentals of Assessment

Fundamentals of Assessment. Todd L. Green, Ph.D. Associate Professor Pharmacology, Physiology & Toxicology . PIES Seminar 2-23-11. Use of terms. Assessments = Observations /measurements Evaluation = interpretations / conclusions . Uses of assessment. Individual learner

gilead
Download Presentation

Fundamentals of Assessment

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Fundamentals of Assessment Todd L. Green, Ph.D. Associate Professor Pharmacology, Physiology & Toxicology PIES Seminar 2-23-11

  2. Use of terms • Assessments = Observations /measurements • Evaluation = interpretations / conclusions

  3. Uses of assessment • Individual learner • Educational action: Feedback • Administrative action: Judgment on advancement • Program • E.g., Sources of variance  Quality improvement • Accreditation

  4. “Unit” of assessment • Single interaction of student and content • Series of encounters (observations by one teacher) • Series of teachers (in a rotation) • Series of teachers + exams (in a clerkship) • Series of clerkship (in a department) • Inter-clerkship (clinical year’s program)

  5. PURPOSE (Why Assess) • Meet goals and objectives? • Were innovations successful? • Determine program strengths and weaknesses • Unanticipated strengths and problems • Growing interest in educational outcomes • “Residency programs should use outcome assessment in their evaluation of program educational effectiveness”1 • LCME, RRC require programmatic evaluation1 1Graduate Medical Education Directory, 2001-2002

  6. PURPOSE (Why Assess) • Competing goals of stakeholders • “Tension between delivering the service and ensuring time for reflection and learning”2 • Help obtain and/or justify resources for your program 2Hayden, 2001

  7. Process: Assessment & Evaluation

  8. What are stakeholders and who are they?

  9. Marshall University Joan C. Edwards School of Medicine Institutional Objectives Patient Care Students must be able to provide patient care that is compassionate, appropriate, and effectivefor the treatment of health problems and the promotion of health. Medical Knowledge Students must demonstrate knowledge of established and evolving biomedical, clinical, epidemiological and social-behavioral sciences, as well as the application of thisknowledge topatient care. Practice-based Learning and Improvement Students must demonstrate the ability to investigate and evaluate their care of patients, toappraise and assimilate scientific evidence, and to continuously improve patient care based onconstant self-evaluation and life-long learning.

  10. Marshall University Joan C. Edwards School of Medicine Institutional Objectives Interpersonal and Communication Skills Students must demonstrate interpersonal and communication skills that result in the effectiveexchange of information and collaboration with patients, their families, and healthprofessionals. Professionalism Students must demonstrate a commitment to carrying out professional responsibilities and anadherence to ethical principles. Systems-based Practice Students must demonstrate an awareness of and responsiveness to the larger context and systemof health care, as well as the ability to call effectively on other resources in the system toprovide optimal health care.

  11. Undergraduate Medical Education DURINGAFTER Logbooks Surveys Hours Placing Graduates Critiques Self assessment Exit interviews Student portfolios Review of write-ups Site Visit Written exams Written exams Performance Evaluation Performance Evaluation Grades/narratives (4th year OSCE, mini-CEX) Attitudinal 4th year medicine performance

  12. Graduate MedicalEducation BEFORE DURING AFTER Self Assessment Logs Surveys Hours Placing Graduates Learner Critiques 360 Degree Evaluation Chart Review Exit interviews Attitudinal Written Exams Written Exams Written Exams Performance Eval Performance Eval Performance Eval GPA Disciplinary Action Narratives Narratives Research/Teaching Global Evaluation National Society Participation

  13. Undergraduate Medical Education

  14. Undergraduate Medical Education

  15. Graduate Medical Education

  16. Graduate Medical Education

  17. Principles of Assessment Fairness Consistency Simplicity Pangaro HMI 07

  18. Principle 1 : Fairness • to society : competence (P/F) • to students : transparency and feedback • to teachers : faculty development and protection Pangaro HMI 07

  19. Principle 2 : Consistency • Validity • Are you measuring what you want? • Reliability - Inter-teacher and inter-clerkship: suitable for high-stakes decisions? Pangaro HMI 07

  20. Reliability versus validity X X X X X X X X X X X X X X X Reliable Valid

  21. Principle 3: Simplicity • How we frame the question? • How we ask the question? • How do we use the answer? Pangaro HMI 07

  22. Strategy • Simplicity leads to acceptance and use • Acceptance leads to consistency • Consistency = fairness Pangaro HMI 07

  23. Framing the question simply • What do we expect of learners (students)? • How do we communicate this to them and to their teachers? Pangaro HMI 07

  24. In order to get the information you need from an assessment and evaluate the results, you have to ask the right questions.

  25. Exercise • Find the person nearest you. Person to the left is “A”. • Person A: In 30 seconds tell the person nearest you what it takes to pass a course/clerkship you are familiar with. How is this assessed? • Person B: In 30 seconds tell the person nearest you what it takes to get the highest grade in a course/clerkship you are familiar with. How is this assessed? Pangaro HMI 07

  26. Discussion Based on the information you have: • Are you able to determine your assessment system’s strengths and weaknesses? • Can you determine whether the system has met the required levels of success for the purpose? • Is it clear how your own program evaluation system will be assessed?

More Related