WiPP WORKLOAD ANALYSIS TOOL Interim feedback on the evaluation 7th September 2007

1. WiPPWORKLOAD ANALYSIS TOOL Interim feedback on the evaluation 7th September 2007

2. Introduction and Overview To update participants on progress to date with the evaluation To report some ‘headline’ findings from phases 1 & 2 To provide an opportunity for feedback and discussion, to help validate the data emerging from the questionnaires and interviews, and inform design of phases 3 & 4 Introduce ourselves Thank people for their participation in the evaluation work so far Explain that the purpose of the presentation is 3-fold: i) to update participants on the work and keep them informed ii) to check back/validate our findings with a different/wider group of participants iii) to contribute to the discussion and dialogue taking place throughout the day, and subsequent planning by the WIPP Team Summarise content of presentationIntroduce ourselves Thank people for their participation in the evaluation work so far Explain that the purpose of the presentation is 3-fold: i) to update participants on the work and keep them informed ii) to check back/validate our findings with a different/wider group of participants iii) to contribute to the discussion and dialogue taking place throughout the day, and subsequent planning by the WIPP Team Summarise content of presentation

3. The Evaluation Programme Preparatory phase (Jan-Feb 07): collating information about the pilot sites, informal interviews with selected respondents Phase 1 (March-April 07): Questionnaire (all pilot sites) Phase 2 (May-July 07) Visits to a sample of pilot sites, and semi-structured interviews with practice leads Explain changes to original timetable for completing evaluationExplain changes to original timetable for completing evaluation

4. The Evaluation Programme Phase 3 (October/November 07): Repeat Questionnaire (all practices) Phase 4 (Nov. 07 – March 08): Follow-up interviews/site visits with selected practices Data Validation (October/November 07): Analysis of workload-related data in small number of case-study sites

5. The story so far… Explain only have time to present some selected ‘headline’ findings and figures. There is more complexity and variation behind some of the issues that will be brought out in the report, but trying to convey the dominant themes Where numbers/% referred to, this is data from the questionnaires Key issues/themes supplemented with comments from the interviews Explain only have time to present some selected ‘headline’ findings and figures. There is more complexity and variation behind some of the issues that will be brought out in the report, but trying to convey the dominant themes Where numbers/% referred to, this is data from the questionnaires Key issues/themes supplemented with comments from the interviews

6. Questionnaire responses 71% (46/65) response rate by end of April Completed by: 89% (41/46) manager 7% practice director/managing partner 4% nurse partner/special nurse practitioner

7. Site-Visits and Interviews 9 sites visited (out of 12 selected). Not a ‘representative’ sample, but included north, south and midlands; urban and rural; large and small; phase 1 & phase 2; different clinical systems 25 interviewees (Practice Manager/Lead GP/Lead Nurse)

8. Why did the practice get involved?

9. Expectations of the Tool … my understanding was that it’s a workload analysis tool with the idea being that you could analyse who’s doing what and also work out whether or not the ‘who’ is the most appropriate person to be doing that work… (GP3) I think from the initial time that it was mentioned the idea was that care [would be] more planned and better organised, but also that the workload would be directed to… the relevant people, and to utilise their skills but also to relieve pressure on general practitioners to free up more appointments for them. So it was making better use of the resources that we’d got that were currently underused (PN5)

10. Clinical Engagement:Nominated Clinical Lead?

11. “The practice’s clinical team is actively engaged with the WAT project” (n=42) 

12. “My practice’s clinical team understands the objectives of the WAT pilot” (n=42)

13. Clinical team’s use of the tool: “It is very limited at the moment, we had a very short presentation of what it was about and how it was working in [terms of] collection of data, read coding etc. by [practice manager] a couple of weeks back” (PN9) [the tool/reports have been discussed] “in 2 partnership meetings, probably for only 15 minutes a time or maybe 30 minutes the first time and 15 the second”. (GP7) On the whole, the practice manager has lead the project, supported by a GP. The wider team was only beginning to get involved at the time of the interviews, usually in the form of having the information presented and discussed at practice meetings.On the whole, the practice manager has lead the project, supported by a GP. The wider team was only beginning to get involved at the time of the interviews, usually in the form of having the information presented and discussed at practice meetings.

14. “It is straightforward to interpret and understand the reports” (n=41) But printing is still a problem, and some of the reports still contain too much dataBut printing is still a problem, and some of the reports still contain too much data

15. “The reports provide accurate measures of clinical workload” (n=39) On reflection, maybe this questions wasn’t sufficiently clear/specific?On reflection, maybe this questions wasn’t sufficiently clear/specific?

16. Accuracy of Reports “I’ve no doubt that the programme or the tool is picking up the episodes that are coded. As I say, the problem is that the coding will vary... It will pick up what’s there to code. What it won’t do is pick up what’s been entered as text. It can’t read between the lines really, that’s the thing”.

17. “The reports address the key measures of clinical workload” (n=37) As above – this might explain the neutral ratingsAs above – this might explain the neutral ratings

18. Domination of Reports with QOF related data [The reports have not been successful in identifying major workload issues] “… because of the… QOF thing, because of just a fog that’s created by our top 40. I think 30 of them are related to QOF” (GP)

19. Have you used the WAT User Guide to help with accessing or interpreting the reports?

20. Has your practice introduced any changes since the beginning of the phase 2 pilot, as a direct result of your involvement in the WAT project? 

21.  ‘Yes’: the practice has introduced changes 

22. Changes resulting from the use of the tool (coding): “… it sounds like we’ve been negative but it’s certainly made me think about the way I enter information and I’ve spoken to my partner… So you know, we will, I think it will improve our general data input, we’ll probably use less of these lazy headings and more specific ones.” (GP3) But note, not all respondents (particularly GPs) feel it is a problems if different clinicians use different read codes (as long as QOF data is accurate)But note, not all respondents (particularly GPs) feel it is a problems if different clinicians use different read codes (as long as QOF data is accurate)

23. Changes resulting from use of the tool (skill-mix) “… we went through and we were able to identify that our nurse practitioner… spent a lot of time doing… things that, you know, either the a lower grade or the HCA could have done, so what we actually have done is we’ve now embargoed 2 of her clinics on a Tuesday and a Thursday and she will only see minor ailments on that day… whereas before it was being filled up with things like people needing BP checks or coming in for travel vaccinations and smoking cessation, routine things… and it’s freed up a lot of space for us…. (PM3)

24. Are you planning to make any changes to clinical workload in the practice over the next 12 months? 

25. ‘Yes’: planning to make any changes to clinical workload, mainly focused on (by staff group): 

26. ‘Yes’ planning to make any changes to clinical workload, mainly focused on (by clinical area): 

27. Which of the following most accurately describes the relationship between these planned changes and your involvement in the WAT Pilot? 

28. “The practice’s use of IT is such that it can be described as paperless” (n=41)

29. IT Literacy: “We’re above the average by far in terms of our computing system. We were in fact one of the early pilot sites in the 80s… when the new computing system first came out. We were piloting it in 1986”. (GP2) “… from talking to other practice nurses or practice managers and I would say that this surgery is probably at the high end of skills with IT and data recording and template development and all that kind of thing.” (PN5) Discuss implications here for roll-out. If this is a self-selected group of practices that are already quite IT literate, how well would the tool be received by others with less interest/skill? Discuss implications here for roll-out. If this is a self-selected group of practices that are already quite IT literate, how well would the tool be received by others with less interest/skill?

30. On the basis of your experience so far, how do you think the WAT … could be improved …? (n=36) [these slides to be refined/ developed a bit][these slides to be refined/ developed a bit]

31. Comments relating to ‘greater sophistication’

32. Comments relating to ‘greater sophistication’ ... identify what the NP see in ... overflow clinics ... Identify split sites ... Identify days of week to see if there are any trends. The QOF data appears to be taking over the actual workload clinical activity. ... Would like to be able to 'drill down' underneath the reports to investigate further…

33. Views on ‘roll-out’ of the tool “I’m sure it will be a benefit to some practices but not all practices. I think practices probably have to be ready to use it, you know, they’ve got to be supported in their reorganisation of their services... I think if you just throw it at practices or make every practice go through it then it will just become an irrelevant exercise which probably won’t be so useful.” (GP)

34. Views on paying for the tool “I think it would depend on how much, because everything does, but it would also depend on the degree to which we could define what reports we got. To have it for free and not to be able to specify your reports is one thing, but to pay for it and not to be able to specify your reports is less palatable, so it depends really. (PM6) I don’t think many practices would want to pay for it because you could probably get the data by other means. I think if practices were going to pay for it they probably wouldn’t want to pay an awful lot. (GP5)

35. “The WAT programme been very well managed and organised so far” (n=40)

36. Summary Observations Some evidence of the tool being used to influence changes to both the use and recording of data, and changes to clinical workload management. At this stage, sites were more likely to report plans or expectations about future changes, than to identify interventions that had actually been implemented. Where changes were taking place, it was sometimes difficult to pinpoint the specific contribution of the tool (different respondents may have different views on this)

37. Summary Observations Clinical involvement with and use of the tool was relatively low, but teams were starting to share the data within the practice, and this was generally well received There were still some concerns about the format, (presentation, printing etc.) of the reports Concerns over the ‘accuracy’ of the reports seem to relate to the recognition that all data are read-coded, rather than suggesting a fault with the extraction tool Feedback on the overall management of the programme by WIPP and the Coordinators was generally very positive.

38. Questions How clear is the long-term strategic vision? What is the tool intended to achieve at the ‘industrial level’? Is this vision shared at a national and local level? Timescale: How long does it really take for before change can be observed in such an initiative? Tool sophistication: is the development strategy optimal?

39. Questions and discussion…