Quality & Safety Markers

2. Quality & Safety Markers • Local process and outcome results presented • Falls, Hand hygiene, Peri-operative harm • CLAB is not presented as already substantial analysis through the CLAB Collaborative which we do not want to duplicate • Excel Workbook presented for ease of use (including local ongoing monitoring if desired) • Following sets out how data are presented

3. Presentation style – Process (1) Space to allow updates Calculation of Process Marker 95% confidence intervals We are 95% certain the true value lies between these two figures Threshold level Description of the national distribution of results to allow comparisons Summary description of position against threshold and national distribution

4. Presentation style – Process (2) Figure 1: Falls risk assessment compared with threshold Actual rates represented by blue bars Thresholds by red line – Basic analogy is that if bar crosses line, the threshold has been met

5. Presentation style – Outcome • Outcome data are presented as monthly time series for the DHB alone • We are not yet confident we have robust risk adjustment between DHBs so inter-DHB comparisons are not given • Time series data is presented straight, as moving averages, and where appropriate as Statistical Process Control Charts or CUSUM V-Masks in order to demonstrate meaningful trends and outliers

6. Outcome – straight time series X axis shows time, y axis variable being measured Shows actual data but “noise” makes any trend hard to discern

7. Outcome – time series with smoothing Addition of LOESS smoothing (the red line) removes the noise and makes trends more easily discernible (in this case suggesting an increase in post operative DVT rate between 2006 and 2009, followed by a stable rate)

8. Outcome –Statistical Process Control (1) • Statistical process control allows the observer to distinguish between changes which are likely to be random variation (common cause variation) and those that indicate something significant has changed (special cause variation). • In order to identify this “Control” limits are set in relation to a set value (usually for our purposes the average for a set period). • This allows identification of both “one-off outliers” and trends • This can be used post-hoc to identify historical outliers which need investigation and understanding or prospectively to measure effects of a planned change • A tremendously useful resource on a fascinating topic can be found at http://www.ihi.org/offerings/Conferences/APACForum2012/Documents/I5_Presentation_Lloyd.pdf

9. Outcome –Statistical Process Control (2) Outliers – months where the sepsis rate was higher than the Control Limit and worthy of further investigation

10. Outcome – CUSUM with V MASK (1) • Cusum is a similar technique to SPC but is measures the cumulative sum of deviations (higher or lower) from the mean value for a given variable over a period under consideration. • By the end of this period these deviations will have cancelled each other out meaning that the Cusum figure will always end at 0. • The V-MASK fulfils a similar function to that of the control limit in an SPC chart. It identifies when changes in recorded results are likely to reflect a genuine change in conditions or process. • However, as change rather than raw data is being plotted, the meaning of direction is reversed on the chart. If the Cusum line crosses the higher V line, this indicates that rates have fallen significantly, and vice versa. • The Cusum is generally regarded as being more effective at identifying small changes or where numbers are small. Consequently for DHB level data we have tended to use Cusum more than SPC • Further information about this approach, including how it was used to track MRSA infections in Scotland can be found at http://www.bentley.edu/centers/sites/www.bentley.edu.centers/files/csbigs/Henderson.pdf

11. Outcome – CUSUM with V MASK (2) By crossing the High V line at this point, this indicates that the number of falls with fractured neck of femur was significantly higher than currently, indicating a significant reduction in recent months