Professor Geoff Shepherd Senior Policy Adviser THE SAINSBURY CENTRE FOR MENTAL HEALTH

1. BASE 2008 ConferenceThe use of ‘Key Performance Indicators’ (KPIs) in the development of local employment services for people with mental health problems Professor Geoff Shepherd Senior Policy Adviser THE SAINSBURY CENTRE FOR MENTAL HEALTH

2. Purpose of the KPIs • To provide an evidence-based framework for the development and monitoring of services to help people in contact with specialist mental health services back to paid employment. • [Also concerned with helping people who have mental health problems who are currently in employment, but this a secondary objective at this stage]. • Aim to bring together research findings and good practice into a single set of indicators that will target development priorities (cf. experience with the NSF) and improve outcomes. • Indicators need to fit within national policies, as well as reflecting the current evidence base

3. Policy background • Currently very strong support at a policy level for increasing access to paid employment for people with mental health problems – a mixture of ‘carrots and sticks’ • Key documents include: • ‘Mental Health and Social Exclusion’ (Social Exclusion Unit, ODPM, 2004) • ‘Disability Discrimination Act 1995’ (amended 2005) • ‘Reaching Out: An action plan on social exclusion’ (SEU, 2006) • Vocational services for people with mental health problems: Commissioning guidance (DoH 2006) • Commissioning Framework for Health and Well-being (DoH, 2007) • ‘Welfare Reform Act’ (2007) • ‘Public Service Agreements’ (PSA targets - HM Treasury, 2007)

4. Research background • ‘Work is good for you ’. Conversely, being made unemployed is ‘bad’ for your mental and physical health (Waddell & Burton, 2007) • Most users of mental health services want to return to work (Grove, Secker & Seebohm, 2005) but most services have not encouraged them much to do so (ODPM Social Exclusion Report, 2002) • Work is one of the key mechanisms for social inclusion and recovery (Shepherd, Boardman & Slade, 2008) • Work is ‘therapy’ and provides a complex range of material, psychological and social benefits that are difficult to replicate in any other intervention. These benefits increase over time

5. The outcome evidence • We now have a demonstrably effective technology for assisting people with SMI to return and maintain positions in paid employment – the ‘Individual Placement and Support’ (IPS) model (Becker, Drake & Concord, 1994) • It is supported by more than 15 RCTs (Latimer, 2008), a Cochrane review (Crowther et al., 2001) and a multi-centre, pan-European trial (EQOLISE – Burns et al., 2007) • We know less about effective job retention • Despite this impressive evidence base, the development of effective services to assist people with severe mental health problems back to work remains patchy and inconsistent

6. The ‘Individual Placement and Support’ (IPS) model (Becker, Drake & Concord, 1994) 7 key principles • Competitive employment is the goal (whole or part-time) • No selection criteria, beyond expressed motivation, i.e. accessible to all who want to work • Focus on consumer preference – ‘fitting the job to the person’ • Based on rapid job search and placement. Minimises pre-employment assessment and training - ‘place-then-train’, not ‘train-then-place’ • Relies on close working between employment specialists and clinical teams • Provides individualized, long-term support with continuity for both the individual and the employer • Includes access to expert Benefits counseling

7. Outcomes from IPS versus ‘treatment as usual’ (Latimer, 2008)

8. Does it work outside America? (Burns et al., Lancet, 2007) A multi-centre RCT in 6 European countries (England, Switzerland, Germany, Netherlands, Italy and Bulgaria) comparing IPS with standard vocational/rehab. services Overall, IPS clients twice as likely to gain employment (55% v. 28%); also worked for significantly longer time Although IPS superior to TAU on all sites, outcomes significantly influenced by local unemployment rates and generosity of Benefits Patients not made unwell by IPS (i.e. no increase in hospitalisations). Total costs for IPS were generally less than standard services over first 6 months

9. EQOLISE - % employed for at least 1 day by location (from Burns et al., 2007)

10. The KPI project group • Geoff Shepherd (Senior Policy Adviser, Sainsbury Centre) • Helen Lockett (Research & Development Manager, Sainsbury Centre) • Jenni Bacon (Administrative Assistant, Sainsbury Centre) • Diane Woods (Associate Director of Commissioning MH & Surrey PCT) • John Ellis (Mental Health Lead Commissioner, Cambridgeshire PCT) • Jonathan Allan (Disability Services Manager, Shropshire CC) • Don Boyle (Oxleas NHS Trust) • Miles Rinaldi (SW London & St.George's NHS Trust) • Andreas Ginkell/Gaynor Chisnall (Independent Provider Forum) • David Carew/Gurcharn Dillon (DWP) • Nick Miller/ Bernadette Oxley (C-SCI) • Anthony Deery/Nicola Vick (HCC) • David Morris/Simon Francis (CSIP)

11. Principles of data collection • Since most of the information will be collected by busy practitioners, every attempt must be made to reduce the additional burden and, wherever possible, use already available datasets. • If new data are required, they should be as simple, brief and as easy to collect as possible. • Each indicator must have ‘provenance’, i.e. be related to specific research evidence, or consistent with current policy. • In order to motivate practitioners indicators must have ‘face validity’, i.e. appear to be potentially useful to them. • They should also get feedback as quickly as possible, in a form which is intelligible and which demonstrates the value of the information

12. The conceptual framework • LOCAL EMPLOYMENT CONTEXT (14 items)Information which characterises the local (un)employment situation. Important background information for commissioners (and providers) • CLIENT INTAKE CHARACTERISTICS (9 items)‘Inputs’ - Who are the services dealing with? • SERVICE EFFECTIVENESS AND QUALITY (8 items)‘Process’ variables - How are the services being delivered? Do they conform to evidence and guidance regarding ‘best practice’ (IPS fidelity scale, Commissioning Guidance) • INDIVIDUAL LEVEL OUTCOMES (10 items)‘Outputs’ - What outcomes are being delivered in terms of ‘resettlement’ and ‘job retention’, from both employment and Health perspectives?

13. So, isn’t this a huge amount of extra work? • SECTION A - We estimate that once commissioners get familiar with extracting data from the NOMIS website (if they aren’t already) it should take no more 2-3 hours per year to collect the information • SECTION B – We assume that most providers collect most of this information on intake anyway. It may mean some redesign of existing systems, but the extra work to routinely collect the data is actually minimal • SECTION C – Most of the data required is derived from annual audits of service quality. We don’t think it unreasonable that service providers should provide a report on what they are doing on an annual basis • SECTION D – This is extra work. Requires quarterly caseload audits + collation into annual reports. However: (a) we will do our best to make this process as ‘painless’ as possible by providing recording templates and (hopefully) instantaneous reports; (b) we intend that the information is seen as intrinsically valuable; and (c) Trusts and LA partners are going to have to do something like this anyway to monitor the PSA targets.

14. Progress so far • First draft of the framework now completed + ‘Glossary’ to define each item and describe how it will be collected. • In collaboration with the NHS Confed MH network, currently conducting pilot studies n=8 (‘Technical sites) + n=9 (‘Implementation sites) • ‘Technical sites’ will receive on-site visits and technical support to collect the dataset (led by SCMH/NHS Confed support) • ‘Implementation sites’ will simply receive introductory explanation + Glossary + telephone support (led by NHS Confed/SCMH support) • All sites will be asked to report on clarity, ease of data collection, perceived usefulness of each item and potential to integrate into existing data collection systems.

15. Pilot sites I. ‘Technical’ (n=8)  Northumberland, Tyne & Wear  Leeds  Gloucester  South Staffs. & Shropshire  North Essex  City & East London  Camden & Islington  Surrey Borders II. ‘Implementation’ (n=9)  South Essex  Kent & Medway  Dorset  Lancashire  Nottinghamshire  Somerset Partnership  Humber  Cheshire & Wirral  Sussex

16. ‘To do’ list • Finish piloting by end of November, complete qualitative quantitative interviews on pilot sites and write-up report • Develop a central database to generate automated reports to give immediate feedback to users of the system regarding their own data and comparative ‘benchmarks’ • Further consultation and recommendations regarding measures of user perspectives on outcomes • Revise and launch framework with a conference early in 2009

17. Potential uses of the information • There are a number of possible uses for the information generated by such a system. These depend on the perspective of individual stakeholders. For example: • Local commissioners – Are we commissioning the best quality services from providers? • Local employment providers – Are we providing high quality services and achieving good outcomes? • Local mental health services (community teams) – Are we contributing most effectively to these outcomes? • Service users and their families – Are we receiving the best possible help from local services? • National regulators and policy bodies – Are local services delivering outcomes consistent with national priorities? • Each stakeholder will have an interest in different elements of the dataset and in different forms of aggregation • However, there will be some commonality and some overlap around common themes

18. For further information Please contact: Professor Geoff Shepherd geoff.shepherd@scmh.org.uk Tel 020 7827 8306 Mobile 0 77 88 721 425