1 / 34

Mega Code

Mega Code. Dr. Nalini Singhal University of Calgary September 2006. Types of Validity. Face Assessment of instrument appearance Does the checklist measure what it is supposed to measure? Content Adequacy of sampling of the domains that should be examined

maude
Download Presentation

Mega Code

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Mega Code Dr. Nalini Singhal University of Calgary September 2006

  2. Types of Validity • Face • Assessment of instrument appearance • Does the checklist measure what it is supposed to measure? • Content • Adequacy of sampling of the domains that should be examined • Does the checklist contain the content it should contain?

  3. Types of Validity • Criterion related (predictive or concurrent) • Correlation between this instrument and some other current instrument or future outcome • Does this checklist correlate with performance on another instrument or some future outcome test? • Construct • Instrument measures the hypothesized skill or ability • Does performance on this checklist vary by location of practice, type of professional or instructor? • Do professionals who follow the sequencing in checklists have better neonatal outcomes?

  4. Challenge of Current Checklists • We do not know whether the current checklists are being used to: • Guide candidates and examiners regarding the spectrum of items to be included or considered in a resuscitation • Score candidates to determine mastery of skill

  5. As a Guide Advantages • Comprehensive • Thorough Disadvantages • Assumes all items are equally important and all need to be done in order presented in checklists

  6. As a Scoring Device Advantages • Comprehensive • Thorough list Disadvantages • Do not provide a method for scoring as cannot tell what is done, not done, N/A • Have not established whether order is important • Have not established item weight

  7. Stages of development • Experts (Neonatologists) drafted a checklist (2002) • NRP committee reviewed the checklist and revised it. Face and content validation by Neonatologists, nurses, and others

  8. Stages of development…cont’d • Checklist sent to 8000 instructors (by e-mail) (2003). 822 reviewed checklist and provided data about: • Criticality of each of the items • Perceived ability to assess students on the items

  9. Stages of development…cont’d • Developed 28 video clips re mega code performance at 5 sites. Recruited 17 instructors who reviewed 2-9 video clips each. Used the newly revised 19 item instrument with scale of 0 (not done), 1 (partially done), 2 (done) to assess performance. Also provided data about ease of assessment. (2004) • Resulted in 20 item checklist and scale of 0= not done; 1 = partially done; 2 = done

  10. Conclusions • High participation levels, fairly representative of the usual participants and instructors • Internal Consistency Reliability: Cronbachs Alpha .69 (overall)

  11. Conclusions ….cont’d • Validity: • Face • Content • Criterion: with written examination evidence. This provides for convergent and divergent validity. (Correlation of .22 between megacode and written exam). • Construct: Ratings not particularly affected by sociodemographic characteristics of student or instructor

  12. The New Mega Code

  13. Do Your Mega Codes Ever Look Like The Three Stooges?

  14. Student Factors • What student factors contribute to Mega Codes going poorly?

  15. Student Factors • Poor Preparation • Did not read the textbook • Did not practice at skill stations • Not Motivated • “Required” to complete the course • Don’t feel it’s part of their job • “I do this all the time, do I really have to show you?” • Other things on their mind

  16. Student Factors • Nervous • Don’t know what to expect • Intimidated by the instructor • Unaccustomed to the “leader” role • Afraid to look bad in front of a colleague or a professional that you supervise

  17. Instructor Factors • What instructor factors contribute to Mega Codes going poorly?

  18. Instructor Factors • Not providing enough practice opportunities • Not giving effective feedback • Incomplete instructions • Intimidated by the learner • Not providing an optimal setting • Poorly planned scenarios

  19. Let’s See an Example…

  20. Strategies For Better Mega Codes

  21. Practice and Setting • Opportunities for structured practice prior to Mega Code • Students can help to “teach” each other and give feedback • Have skill stations available for “off shift” staff to practice before course • Set up your Mega Code in a realistic setting, not the conference room table

  22. Demonstrations • Demonstrate a Mega Code • Interactive Mega Code (a group practice) • Instructors demonstrate a Mega Code in “real time”

  23. Expectations • Make them clear • The Mega Code is a performance evaluation. • The Mega Code is not a practice session. • If students are unprepared, stop, give constructive feedback, encourage more practice, offer another Mega Code opportunity.

  24. Mega Code 2006 • You have a standardized set of instructions to read to the student(s) before the Mega Code starts. • The Mega Code is scored.

  25. Mega Code 2006 • Skills from each chapter that the student completes, within their scope of practice, must be tested in the scenario. • All tested skills must be demonstrated. • Students cannot “talk through” skills.

  26. Mega Code 2006 • Students completing the medication lesson must draw up “epinephrine” correctly. • Mega Code occurs in “real” time. • If a student fails the Mega Code twice, recommend additional practice and come back another time.

  27. Scoring the Mega Code • Mega Code is scored • 0 points = not done • 1 point = done inadequately or out of correct sequence • 2 points = done well and in correct order

  28. Scoring the Mega Code • Minimum Mega Code = PPV and Chest Compressions • Minimum 14 items (28 points) • Maximum 19 items (38 points) • The number of points varies depending on the complexity of the scenario you give.

  29. Scoring the Mega Code • You must achieve 2 points for each of the following 5 items: • Checks bag, mask, and oxygen supply • Indicates need for PPV (apnea or HR < 100) • Provides PPV correctly • Indicates how to correct chest not moving if there is no improvement in HR • Correctly provides chest compressions (HR < 60) • Must achieve 85% of total possible points

  30. Mega Code Assessment (Basic)

  31. Thank you

More Related