Clinical Performance Development in NHS Direct 2003-4

1. Clinical Performance Development in NHS Direct 2003-4 Dr Andrew Lee NAGPC Annual Conference March 2004

2. Clinical performance indicators • Standard national reports replaced individual site versions • Reported at national, network, site, nursing team, and individual nurse level • Separate 0845 and OOH Clinical performance targets • Sorting of symptomatic calls • Use of NHS CAS

3. Gold Standard Sorting Study

4. Aims • Establish a benchmark of existing NHS practice in clinical risk tolerance for primary presentations (first point of contact) by telephone • Identify and quantify any appropriate difference in the sorting profile for calls made to access GP OOH services and that for other NHS Direct calls • Use the results to inform target setting for NHS Direct clinical performance indicators

5. Gold Standard Sorting Study • First Stage - Benchmarking • 113 GPs from England and Wales • Random sample of 600 NHS Direct calls • 300 ‘0845’ and 300 ‘GP OOH’ • Clinical details of each call reviewed independently by 10 reviewers

6. Gold Standard Sorting Study • Second stage – Reviewing the Differences • Cases with significant disagreement between reviewers and NHSD identified • Consensus review of these by senior NHSD medical and nursing clinicians taking account of : • NHS guidelines, clinical consistency and context of service model

7. Gold Standard Dispositions • Dispositions used to calculate Gold Standard: • NHSD actual dispositions where consensus amongst reviewers same • NHSD actual dispositions where no consensus at all amongst reviewers • Dispositions chosen by senior NHSD clinicians in Consensus review stage

8. GP consensus? • Unanimity amongst reviewers on level of care in only 7% of cases • At least 2:1 consensus amongst reviewers on a level of care in 46% of cases • At least 2:1 consensus amongst reviewers on whether referral to urgent care (999/A&E/GPOOH) needed or not in 74% of cases

9. When GP consensus re Urgent referral/Not • High concordance with NHS Direct • NHSD actual disposition was same as reviewer consensus in 78% of cases • Gold standard is same as reviewer consensus in 96% of cases

10. Use of NHS CAS • Compared the accuracy and risk of the three patterns of assessment: • Algo used and followed • Algo used but a different disposition chosen • No algorithm used • Risk measured as ‘potentially critical misses’ i.e. a Gold Standard disposition of Emergency or Urgent Care (999, A&E, GP 4 or 12) with an actual NHSD disposition below this level (in hours care or self care)

11. NHS CAS results • Algorithm content safe • Algorithm delivers gold standard sorting or nearer to it than altering in over 80% of use • Not using an algorithm produced a rate of ‘potentially critical misses’ nearly three times higher than when an algorithm was used whether the recommended disposition was followed or not

12. Comparison with other studies ‘SWOOP’ study of nurse-led OOH triage (1997): 49.2% referred to GP OOH service (calls before midnight) NHSD Gold standard for GP OOH (combination of GP within 2 hours and GP within 6 hours) = 43% (+ some ‘GP next day’ will go to OOH at weekends) Sheffield MCRU Appropriateness Study (2003): 13% ‘overtriage’ and 1% ‘undertriage’ (equivalent to a net reduction in urgency of 12%) NHSD Gold standard = reduction in emergency&urgent care 9.7% (0845) and 11.1% (OOH)

13. NHS CAS developments • Disposition wording changed to make timeframes consistent with OOH access standards (1, 2 and 6 hours) • Twice yearly major clinical content version releases ongoing • Version 12 (April/May 2004) contains first Gold Standard related revisions

14. Summary • GPs have difficulty agreeing on appropriate sorting • NHS Direct’s sorting is not very different from what GPs can broadly agree on • NHS Direct has a good safety record • NHS Direct’s sorting can improve • NHS Direct is taking actions to achieve that improvement • NHS CAS will help nurses deliver that improvement