The COGPED Completed Cycle Audit

1. The COGPED Completed Cycle Audit Tim Swanwick Director of Postgraduate General Practice Education London Deanery

2. what are your issues? • in pairs (5 minutes)

3. what are my issues? • really useful assessment • very clear marking schedule • no one should fail • I don’t want to have to advise on so many resubmissions!

4. Reason for choice of audit Criterion/criteria chosen Standards set Preparation and Planning Data collection 1 Changes to be evaluated Data collection 2 Conclusions Potential for change & relevant to practice Relevant to audit subject and justifiable e.g. current literature Target towards a standard with a suitable timetable Evidence of teamwork and adequate discussion where appropriate Results compared against standard Actual example described Comparison with Data collection (1) Summary of main issues learned marking schedule

5. exercise • in threes • “I’m going to send this in, what do you think?”

6. reason for choice of audit • potential for change & relevant to practice • potential for change - can do something about problems • relevant to practice – hospital OK – e.g. emergency admissions • common errors • over-complicated • too ambitious • no potential for change • irrelevant to practice

7. criterion/criteria chosen • relevant to audit subject and justifiable • common errors • no understanding of a criterion • not “inclusion criteria” • too many criteria • nested criteria • criteria confused with standards • relevant – is it about the chosen subject? • justifiable – is it a suitable thing to measure?

8. what is a criterion? • a criterion is a statement of quality • a criterion is meaningful and measurable • “all” and “should” rule • a criterion requires a measurable aspect of care , a quality marker and a “population” The last recorded BP of all patients with diabetes should be130/80 or less All telephone calls into the surgery should be answered within five rings

9. examples of criteria (exercise) • In threes, give an example of a criterion in each of the following categories • chronic disease • prescribing • management of acute conditions • practice administration • screening

10. standards set • target towards a standard + suitable timeframe • evidence that might be used • NHS or local targets • guidelines • expert advice • focus group • benchmarking • practice data on other audits • common errors • confused with criteria • not justified • 100% usually over-zealous

11. what is a standard? • a figure, usually a % • every criterion must have one • must be justified

12. preparation & planning • evidence of teamwork and adequate discussion where appropriate • teamwork is an absolute !! • reasonable description of process • common errors • “I did it my way…”

13. data collection 1 • result compared with standard • a grid makes life easier • common errors • “simply, too many notes” • data not related to criteria • data not compared to standard • discrepancies unexplained

14. changes to be evaluated • actual examples described • short term and.. • … long term: “what will happen in the practice when I’m gone?” • group work • systems, practice procedures etc • common errors • no changes described at all • no sustainable changes: “I wrote a letter to all the patients…”

15. data collection 2 • comparison with data collection 1 • again a clear grid is a big help • results may be better, worse or the same

16. common errors data 2 not collected in same way as data 1 only the patients subjected to short term intervention counted results not compared comparison not adequately discussed data collection 2 (continued)

17. conclusions • summary of main issues learned • specific points about the audit • general points about the process • common errors • rare to fail just this • inadequate discussion • unjustified assertions

18. results During the the period 1.4.02-31.3.03 • 214 GPRs passed SA in London • 12 through NPMS (5.6%) • 202 through audit • 22 of these were resubmissions (c.10%) • 2 failed during this period (c.1%)

19. referral • there’s no failure, only feedback • the vast majority can be rewritten • “how to do it” guide on deanery web site www.londondeanery.ac.uk • guidance on resubmission (only) from TS if required

20. exercise • “my audit’s been referred – help!”

21. questions and answers?

22. National Project Marking Schedule

23. National Project Marking Schedule (NPMS) • Approved by the JCPTGP from April 2000 • A project: • addresses a defined problem • is related to previous work • presents qualitative or quantitative findings • interprets these findings • draws conclusions from the evidence presented

24. NPMS • competencies tested: • the ability to construct a logical argument • the ability to communicate in written English • the ability to plan and sustain activity over time

25. NPMS • range of acceptable work includes: • a small research study e.g.questionnaire, notes review, interview study • a literature review • a case study • a proposal for a new service • a discussion paper

26. NPMS marking schedule • six criteria for the project • each criterion marked from 0 – 5 • a pass will score 18 or more • each criterion must score 2 or more

27. NPMS marking schedule (continued) • aims/question/problem clearly stated • relevant literature cited • method appropriate • relevant findings presented • discussion appropriate • conclusions appropriate

28. NPMS 2002/3 • 7% of all registrars • April 2002- March 2003 • 126 audits • 87% first and second level passes • 13% referred • all marked by the Yorkshire Deanery

29. NPMS - The Future • National Project Marking Schedule to become “National” • Audit may become subsumed into trainer’s report • Marking panels in all deaneries • Timescale … 2-3 years • ?A role within the MRCGP