1 / 20

Implementation: Pilot testing – and further development – a critical next step

Implementation: Pilot testing – and further development – a critical next step. David F. Ransohoff MD Departments of Medicine and Epidemiology University of North Carolina at Chapel Hill. Implementation steps: •certify organizations. from p146,7.

latoya
Download Presentation

Implementation: Pilot testing – and further development – a critical next step

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Implementation: Pilot testing– and further development – a critical next step David F. Ransohoff MDDepartments of Medicine and EpidemiologyUniversity of North Carolina at Chapel Hill

  2. Implementation steps:•certify organizations from p146,7

  3. Implementation steps:•certify organizations •what’s missing: how to measure “how trustworthy” is individual CPG from p146,7

  4. Howtrustworthy is a CPG? • a. How is “trustworthy” measured(scale) •existential (yes/no)•ordinal (very trustworthy, moderately..., not)•numerical • b. How are the 8 “standards”:•measured individually •‘combined,’ to decide (a) “how trustworthy”? • Doing (a) and (b) is ‘scale-making’

  5. Scale-making is commonly done e.g, Apgar Score, or SF-36 These scales “components” have multiple categories. For many scales “coding manuals” are developed.

  6. Howtrustworthy is a CPG? Example: What happens if we apply “standards” to CRC screening guidelines,that were a “call-out”: can we tell how trustworthy each is?

  7. How trustworthy is a CPG?(CRC MSTF 2008) • ... according to a panel chair:“[T]he process ultimately became politicized and, according to one participant, resembled “sausage-making.”’ • -Cancer Letter, Oct 12, 2008

  8. Howtrustworthy is a CPG? Current Instructions to apply standards

  9. Howtrustworthy is a CPG? Current Instructions to apply standards Current instructions ask only: “Is everything perfect.” There’s no “threshold,” and no details for individual standards. “Pilot testing” could show whatwould user (CMS? BC/BS? Kaiser?) think or do; how would they use? And then are YOU(IOM) satisfied with how they use them? Do current standards solve the problem we’ve got in this field?

  10. Why might we need a more-explicit way to measure “how trustworthy”? • 1. Even if you “certify organizations,” must also be able to judge organizations’ products. (e.g., pharmaceuticals) • 2. Many guidelines will be made by “uncertified organizations” (or do you expect all to stop?). • 3. So a user then needs to know how to judge a CPG – individual elements, ‘composite score’. • 4. #3 was IOM’s original charge. • 5. If this were a pharmaceutical, would it be “rolled out” without #3, and without testing? (Does this solve problem in field?) • 6. We need strong checks-and-balances in an environmentwhen it’s hard for professionals to be “professional.”

  11. What a “profession is (via Louis Brandeis and Sam Thier) • Why Sam Thiertalked about this all the time: ‘Society grants privileges to professions (title, income)... in exchange for ‘pursued largely for others’. If a profession doesn’t act right, privileges will be taken away.

  12. Example of one profession’s concerns re future • (AGA Institute Future Trends Committee conference, 2006)

  13. Example of one profession’s concerns re future • (AGA Institute Future Trends Committee conference, 2006) • ‘The Doomsday Conference’ Separate example: May 9, 2011 AGA Plenary, on screening: “Disruptive Technology”

  14. Checks and balances • 1. “Professional organization” wears 2 conflicting hats:1) clients’ interest comes first (Brandeis; Thier)2) professionals’ economic interest comes first • 2. Intellectual conflictexists apart from economics: people “believe in” what they do. (Fowler FJ, et al. Comparison of recommendations by urologists and radiation oncologists for treatment of clinically localized prostate cancer. JAMA 2000;283:3217)

  15. Checks and balances • 3. Lessons from evolution of “composition” of ACS-MSTF(Multi Society Task Force)•# generalists/methodologists in MSTF author group 1997: 4 (RHF, FG, CDM, SHW) Gastroenterology 1997;112:594

  16. Checks and balances • 3. Lessons from evolution of “composition” of ACS-MSTF(Multi Society Task Force)•# generalists/methodologists in MSTF author group 1997: 4 (RHF, FG, CDM, SHW) Gastroenterology 1997;112:594Why, btw, was MSTF formed? (Sid W; AGA; May 9)

  17. Checks and balances • 3. Lessons from evolution of “composition” of ACS-MSTF(Multi Society Task Force)•# generalists/methodologists in MSTF author group 1997: 4 (RHF, FG, CDM, SHW) Gastroenterology 1997;112:594 2003: 2 (RHF, SHW) Gastroenterology 2003;124:544 2008: 0 Gastroenterology 2008;134:1570

  18. Checks and balances • 3. Lessons from evolution of “composition” of ACS-MSTF(Multi Society Task Force)•# generalists/methodologists in MSTF author group 1997: 4 (RHF, FG, CDM, SHW) Gastroenterology 1997;112:594 2003: 2 (RHF, SHW) Gastroenterology 2003;124:544 2008: 0 Gastroenterology 2008;134:1570 • The Task Force was “multisociety”; but over time, dominated by subspecialists and lost all generalists/methodologists. • Each iteration of the guidelines was “less trustworthy”; by2008 “very not trustworthy”. • Evolution reflect “inclination”, and power of economic hat. • Anecdotes about guidelines-making are ugly, disturbing, and cleverly hidden. Not ‘good faith’.

  19. Checks and balances • The challengeGuidelines that are in clients’ interests:•really need the expertise hat•really need to avoid the economic hat. • Do current standards deal strongly-enough with this tension? I think not.

  20. Conclusion and Suggestion • 1. IOM (someone) needs to further develop detailsof standards and of implementation, to provide effective checks-and-balances. • 2. Jefferson and Madison didn’t do their work “suddenly.” It required time/experience to develop checks and balances. Can we establish “process”, mechanism to test, develop standards, “trustworthy” and see if they look like they are solving problem? Who does this? • 3. Guidelines-making is a central issue in “professionalism” in our era; a neutral, professional, authoritative group (IOM?) needs to do/direct this. Can anyone else?

More Related