Workforce Assurance Tool Training Guidance Pack - DRAFT

Workforce Assurance ToolTraining Guidance Pack - DRAFT NHS North West Friday, 11 January 2013

Contents Module 1 Background and Context • The need for effective workforce assurance • Assurance framework overview • Current assurance challenges and how the system will meet these challenges • The intended benefits of the system • Principles underpinning the system • Key components of the system • How the system can by used by different groups • Expected levels of efforts and return to use the system • The data that the system will have access to • The main components of the tool • System views, navigation and functionality • Triggers, metrics and data thresholds • Access to the system • User exercises • Key Questions and Answers Module 2 The system and how it will work Module 3 Intro to the Tool Module 4 FAQ’s

Module 1 Background and Context

1 – Background 3 – Tool The need for effective workforce assurance • Right people, right place, right time • An appropriately sized, skilled, organised and motivated workforce is critical to delivering efficient, quality & safe services. • b. Managing Risk • Workforce assurance in the NHS is the process of managing risk and assuring that the composition of the workforce can deliver cost effective, safe and high quality care. Regardless of reforms to the NHS that result from ‘Liberating the NHS’, the need for assurance processes that review and challenge workforce plans is likely to remain. • c. Quality & Safety • Who leads this process in the new system architecture is yet to be determined but it is critical providers are given the tools to review the impact of workforce changes on quality and safety for themselves – this is an important part of the SHAs’ legacy. • d. Manage Change • Assurance is needed due to the pressures and inherent risks associated with large scale workforce change from: • Tough financial settlements – workforce expenditures represent 70% of the total NHS spend and although in recent years workforce numbers have increased inline with overall spending growth, there is now a very significant productivity challenge related to achieving the overall £20 billion national QIPP target. • Increasing demands for services, and the challenges and focus resulting from the Robert Francis Inquiry into Mid-Staffordshire NHS Foundation Trust. 2 – System 4 – FAQ’s

1 – Background 3 – Tool The need for effective workforce assurance • Workforce assurance has impacts at all levels of the system: • on providers to improve their workforce planning, monitoring systems and impact analysis of their increased productivity efforts. Fundamentally assessing their workforce changes and the impact on quality and safety • on education commissionersto make better education commissioning decisions • on service commissionersto undertake assurance with respect to quality and safety of provider delivery plans • on SHAs with assurance responsibilities to exercise appropriate judgement given the system pressures. 2 – System 4 – FAQ’s

1 – Background 3 – Tool The assurance framework The Workforce Assurance Framework provides a method for achieving system assurance that the planned and available workforce can deliver safe and quality services. 2 – System 4 – FAQ’s • The Framework provides a means to identify both current workforce issues and provides an early warning system when reviewing projected workforce changes. • The assurance framework also reflects the key lines of enquiry agreed by the DH Operations Board1 • The Department of Health are currently developing a national framework for workforce assurance. • This document describes an approach that will support NHS organisations in implementing this framework in a consistent, transparent and highly automated manner. • NB. Whilst the framework is, to a limited degree, implemented manually there are significant advantages in terms of automation and empowerment for Trusts to embedding it within a tool. 1 – Key lines of enquiry referenced in DH KLOE on Safety and Quality Assurance of Workforce Changes

1 – Background 3 – Tool The need for a robust system to support workforce assurance Expertise from 8 of 10 SHAs and the Department of Health have been drawn upon to support system design and to address the current limitations associated with attempting workforce assurance using static data and manual processes. The key challenges identified by this group in relation to workforce assurance and the related desired benefits of the system are detailed below: 2 – System 4 – FAQ’s Current Challenges Solutions from the tool • Much of the data collected is currently retrospective and not forward looking • The tool will use trend data to project how the picture might change over time which can then be compared to actual performance • Process, dashboards and metrics collected are different in different regions • Good practice adopted consistently across Trusts and regions with a collation of all existing metrics • Individual indicators are often looked at in silos • Enables consideration of the importance of interrelationship between indicators and triangulation of workforce, activity and finance • Current processes are not supported by robust statistical analysis • Assurance processes underpinned by detailed information base and statistical analysis including detailed literature review • Current processes are labour intensive as mainly manual processes • Automation reduced manual data manipulation and handling burden • Currently workforce plans go backwards and forwards between Trusts and Assurers, increasing timescales / tensions • Greater visibility of assurance expectations between Trusts and Assurers – a shared tool that can be also be used by individual Providers

Module 2 The system and how it will work

1 – Background 3 – Tool Better use of technology will help achieve this • During the proof of concept stage, a series of challenges that were most appropriately resolved through the use of technology were: • The need to collate and manage huge amounts of both structured and unstructured data – this is currently extremely time consuming both for Trusts and SHAs; • The need to display dynamic / regionalised views of this data in a consistent way; • The need to compare and contrast on an equal footing across all areas of the system; • The need to reduce the level of bureaucracy involved in metric analysis and performance management; • The need to retain and access a clear audit trail for all analysis and action; • The need to enable the assurance process and outputs to be “agile” enough to respond to changes in the operating environment; • The need to support workflow and collaboration (again, with a clear audit trail); and • The need to test robust workforce scenarios 2 – System 4 – FAQ’s

1 – Background 3 – Tool A different approach to assurance • What is different? • A systematic approach enabling rigorous, structured evaluation • Forward and historic looking analysis • A statistically robust underpinning for assurance analysis (correlation analysis will help set metric thresholds and identify new areas in need of monitoring) • Analysis of correlations and relationships between metrics • Escalation routines to enable consideration of escalation in structured way • This enables differing groups of users to: • Provide a tool and legacy for trusts so that they can self-assure and pre-emptively respond to concerns of assurers • Call on robust information and analysis as support for key decision making • Take a more integrated approach to assurance activities (looking across activity, finance, quality and workforce) • Forecast, and indentify correlations within, data to suggest expected trajectory and to see how seemingly unconnected triggers interact within an organisation • Create and drive top table discussions • Change the nature of assurance engagement with Trust Boards and Management Teams • Justify decisions 2 – System 4 – FAQ’s

1 – Background 3 – Tool What the tool is intended for... The Tool IS NOT intended to be: • A tool to publish information • A performance management tool • A data collection tool; existing data is used • A directorate, divisional and professional assurance tool as other resources are available for this 2 – System 4 – FAQ’s The Tool IS intended to provide: • The workforce element that will link with wider system level assurance • Triangulation between workforce, activity and finance data • A national platform to workforce assurance that can be adapted as appropriate for each SHA • Best practice approach that is built on existing published literature • An approach that recognises that assurance criteria will change and be refined over time

1 – Background 3 – Tool Underlying principles of the tool Fundamental Tenets Effective workforce assurance is a critical component of helping ensure the ongoing efficient and effective delivery of services across the NHS system. Organisations should be supported to be rigorous in their self assurance processes, to improve outcomes and reduce the need for external assurance. By using data appropriately, users can be supported to make better assurance decisions. This is a decision support tool, not a decision making tool. Consistent high levels of assurance across regions. Considers a cross section of demand, activity, quality, finance and service information rather than considering workforce alone. Balances forward looking assessments with previous performance: i.e. Historic dashboards are a useful input but do not tell the whole story. To minimise effort, Nationally available data will be used wherever possible to reduce the administrative burden. Differences in data quality and availability need to be allowed for and highlighted within the system. Throughout the life of the tool, the system will be actively managed so that as new national datasets become available each period they are uploaded into the tool. 2 – System 4 – FAQ’s Workforce Assurance in Practice Data usage

1 – Background 3 – Tool How it will be used by different audiences 2 – System 4 – FAQ’s

1 – Background 3 – Tool Provider worked example Scenario 1: Workforce Planning Scenario 2: In-year review Plan • Trend projections on dashboard highlight that deviations from plan for one team within the Trust could be indicative of potential quality and patient safety risks if left unchecked. • Trust uses projection information to review potential risks and inform internal risk prioritisation processes. • If the Trust deems that action is required, they use this information to support engagement between the team’s clinicians and service managers as to nature and extent of concerns. • Appropriate action taken within that team, including development of mitigation plans and reworking forecasts as required. • A Trust works with clinicians, service managers and finance to develop a workforce plan. • Before finalisation, Trust uses the tool to review the plan. This enables the Trust to pinpoint areas of risk and highlight where there are tensions between activity, finance and workforce plans. • The Trust uses the comparative information, threshold and trend information from the tool to engage in a data driven conversation with clinicians and service managers as to whether alternative approaches need to be considered. • In some areas the Trust concludes that despite the indication of risk their plans are in fact sound. In other areas the Trust decides to enhance their plans. • The Trust submits their amended plans with a data driven narrative highlighting why they believe they are sound in perceived risk areas. • The assurer receives the plans and can consider in light of the Trust’s considered risk assessment. 2 – System 4 – FAQ’s Identify Review Consults the dashboard to view overall assurance level Review Assess Engage Act Act Submit Assure Organisations can also use this tool to review current workforce considerations and should always use the tool in conjunction with more qualitative intelligence

1 – Background 3 – Tool Commissioner / assurer worked example Scenario 1: Commissioner demand management Scenario 2: Assurance during transition* Plan • Commissioning team decide to test the effect on workforce in Trust A from their demand management QIPP plans. • Commissioners use the tool to model the activity and income changes on Trust A and understand any workforce, quality or safety issues this may cause. • The Commissioner chooses a scenario which gives the least risk, as seen by the relative score from the dashboard, and discusses impacts with Trust A. • Trust A uses the same scenario and discusses the dashboard score with managers and clinicians to understand the deliverability of the QIPP plan. • The Commissioner and Trust A agree the scenario for demand management and the potential effect on the Trust and target decommissioning and resource toward delivering the target together. • Assurer chooses the organisations to be assured within the tool and opens dashboards for each of these organisations in a separate web browser window • Assurer reviews comparability of potential risks across the group, including assessing trends and changing performance over time. • Assurer prepares guidance on key comparable risks displayed by the dashboard and distributes to providers while also offering support and resource to help mitigate potential risks. • The assurer continues to monitor month by month performance against the key risk areas and works with providers to target mitigation. 2 – System 4 – FAQ’s Identify Analyse Review Consults the dashboard to view overall assurance level Discuss Engage Evaluate Monitor Deliver * Assurance will not just be limited to transition but transition is used as an example due to the heightened risk environment

1 – Background 3 – Tool Data within the system Data included in the initial build • Activity: Health Episode Statistics, Quality Outcomes Framework, Unify, forward looking activity plans, Information Centre • Workforce: ESR, Census, forward looking workforce plans • Quality/Safety: STEIS, Unify, NHS Patient and Staff Survey, IC Complaints data base, NPSA Incident and CAS systems, HPA, Monitor • Finance: FIMS, forward looking finance plans, Monitor 2 – System 4 – FAQ’s Sector specific data that would be useful in the initial build but are not available • The acute sector and national datasets provide the richest streams of data for assurance as this has been the focus of reporting and data collection for the NHS / DH in recent years • Mental Health and Community services will be partially covered due to the absence of quality national data that will limit the robustness of framework (non workforce information primarily) • Primary care data is not robust enough to provide for primary care workforce assurance and therefore primary care data will be looked at in relation to Provider Assurance and give a perspective on the local health economy • The following sectors will not be covered in the system and therefore related data will note be included: • Dental practices • Ophthalmology • Pharmacy • Private sector • Social Care Sector specific data that will not be included

1 – Background 3 – Tool Input required from Trusts The tool has been designed to require zero / minimal input from Trusts, and can operate with or without forward workforce plans. However, with these plans the value delivered from the tool with be far greater. 2 – System 4 – FAQ’s Benefits Received With Plans: This will include comparison against plan functionality and the ability to use trend projections to test the quality and safety implications of the plan. The scenario modelling element of the tool can be done without providing visibility of the plan to assurers. Without Plans: the tool will provide triangulation, comparisons, and projection using national data sets covering workforce, quality, safety, activity and finance. Adding workforce plans, with summary finance and activity information Level of Input • Do Trusts have to input data to the tool? • All data will be provided centrally with two potential exceptions: • If Trusts wish to upload a workforce plan / scenario into a private part of the tool for review (i.e. Before I submit my plan, can I check if it will flag up any major areas of concern at the SHA?), then this will be possible at the users discretion; and • Where a regional collection process does not exist, then approved plans may be uploaded by Trusts at their own discretion.

Module 3: part 1 Introduction to the Tool

1 – Background 3 – Tool Key Components 2 – System 4 – FAQ’s Workforce Assurance Portal My Qualitative Indicators My Plans My Assurance Logs My Flags My Notes My Threads My Actions My Analysis My Resources Help Organisation Dashboard Supporting Materials Community Data Sources Metric Dashboard FAQ Workforce Dictionary

1 – Background 3 – Tool My analysis • Regional User View • Purpose: • This component contains the primary dashboards within the system that operate at regional, organisation and specific metric levels. • Each dashboard highlights areas of potential workforce risk and provides functionality to allow users to drill into risk areas to see more detail. • The dashboards also allow users to view the underlying data, and perform comparisons against peer organisations. • Key features: • Drill down functionality to enable users to see more detail • Further information can be found on each metric, including how the metric has been calculated. • The source data indicators can also be viewed. • Links to enable users to jump straight to actions from the dashboards. • Geographic mapping at a regional level 2 – System 4 – FAQ’s • Organisational Dashboard • Metric Dashboard

1 – Background 3 – Tool Regional User View 2 – System 4 – FAQ’s The organisation summary panel shows a selection of key indicators for the trust Clicking on the assurance cheese opens an Organisation summary panel The section at the bottom of the organisational summary panel provides quick links to the organisation dashboard and actions

1 – Background 3 – Tool Organisational dashboard • It will initially present the scores against each assurance group, using the navigation users can drill down into detail on specific metrics 2 – System 4 – FAQ’s Summary scores are colour coded and displayed against each assurance category Click in any of + or – signs to expand / contract the section

1 – Background 3 – Tool Metrics • Opening up different assurance categories allows you to drill down to metric information. • Clicking on any of the icons provides further information on the metric. • The icons next to each metrics indicate: • Uploaded: the values displayed have been directly uploaded from another source • Calculated: the values displayed have been calculated from other information uploaded. 2 – System 4 – FAQ’s • Things to note on adding of scores: • If there is only 1 metric then this will be the same as the trigger score (e.g. CIP example) • If there is more than one metric in the trigger then these wont necessarily equal the trigger score due to the weighting Weighting Actual value of this metric Contribution of this metric to the Assurance Group score

1 – Background 3 – Tool Indicators, Metrics and Thresholds Example: Indicators, Metrics and Thresholds • Indicator (590 in total) • The base level of data used to make metrics • This data comes from various source data sets • Metric • Metrics display information about an aspect related to workforce • Metrics can also be direct (taken directly from one indicator) or compound (calculated by combining more than one indicator) • Threshold • The threshold is the level at which metrics turn green, amber or red, indicating the risk associated with each metric • The threshold is different for each metric and based on the value needed for them to be considered high risk, and turn red, or low risk, and turn green • The threshold can take the form of a scale or range 2 – System 4 – FAQ’s An indicator could be the number of midwives, or the number of births. The data from these indicators is used to create different metrics. Metric e.g. Midwife: birth ratio Target Value Intervals from Target Value The metric will be assigned a risk level depending on the threshold. Some thresholds are scales where one extreme is good and the other is bad. Other thresholds are ranges where there is an ideal level and anything either side increases risk. In our example a threshold would be a scale, and less than e.g. 3 midwives per birth is ‘high risk’. Intervals from Target Value Intervals from Target Value

1 – Background 3 – Tool Score, Weighting and Aggregation Example: • Score • Each metric within the Assurance Map is given a score indicating the level of risk (low, medium, high) • These scores add up, once normalised and weighted, to generate the overall assurance group score and indication of risk • Weighting • Each metric is assigned a weight based on six categories: • Patient safety • Patient satisfaction • Staff safety • Quality Improvement • Financial control • Staff Health and Wellbeing • Each category is sub weighted, and the score in each category is used to come to a final score for the metric • Aggregation • Scores can then be aggregated to give a score for each Trigger, Trigger Group and Assurance Group. Each Trigger and Trigger Group are also weighted Scoring Nurse to bed ratio rating = Red (Based on set threshold values) Nurse to bed ratio normalised score = -0.8 2 – System 4 – FAQ’s Trust A Nurse to bed ratio = 0.1 Calculation

1 – Background 3 – Tool How the scores add up The scoring and weighting of metrics should be seen as an overall process with distinct, time-bound, activities and linkages. 2 – System 4 – FAQ’s Scoring Weighting Adjustment Aggregation Using established and new thresholds we normalise threshold values and assign a comparable score to all metrics within the Assurance Map Each individual metric score is then weighted by a defined criteria to enable an hierarchy of importance to be constructed All weighted scores can then be subsequently adjusted to account for data coverage and confidence based on statistical analysis. The aim is to account for data quality and quantity problems and show how “reliable” the underlying data used in metrics is Scores can then be aggregated to give a score for each Trigger, Trigger Group and Assurance Group • Things to note: • Scores come in three forms • Raw scores provide a base level indication of the perceived risk • Weighted scores also indicate perceived risk, but some metrics are identified a potentially larger impact on the organisational workforce • It is the weighted score, adjusted for data quality so we know which data we can rely on

1 – Background 3 – Tool Inference Mechanisms • Certain data sources are incomplete for all periods. This could be due to timing or data quality issues. To solve this problem the system will determine values it needs to fill and select from a variety of methods for inferring missing. This can be forward looking or backward looking, using a variety of methods. These include: • Use last known values • Leave blank • Forecasting - Trends are extrapolated from the data. The type of data set will determine the model/method used to extrapolate. Forecasts for 3 and 12 months are then computed • Pro-rating – Use linear interpolation to estimate the data between two points. This may include using a planned future value compared to an actual, allocating the difference equally across the time periods • Relationships - The data will be examined to reveal relationships between metrics. Relationships derived from this process will then be fed in to forecasting models as inputs • Custom calculations (Not used currently) – Use a custom specified mode, calculation or alternative data source to replace the data 2 – System 4 – FAQ’s Data that requires inferring for the past. CHOOSE METHOD: Data source provided with gaps Use previous value Package is called and data is inferred / delivered Use forecast Data that requires inferring for the future. Use replacement Etc... System infers and returns data Select data where the value is “null” based on comparisons with the expected data set (e.g. Open periods, actuality) Based on the Indicator Type, chose which approach to use

1 – Background 3 – Tool My Actions • Purpose: • Whereas the My Analysis section primarily allows you to view information within the system, the My Actions component allows you to undertake six different types of actions within the system • Key Actions: • Logs: regional assurers can automate their process of monthly log recording • Threads: records communications and exchange between Trusts and regional assurers can be recorded and tracked • Quantitative Indicators: can be recorded, viewed and edited • Notes: user specific notes can be recorded which can only be viewed by the author • Flags: alerts and flags can be viewed and overridden. • Plan data: trust plans can be uploaded into the system • Reports: reports can be viewed and searched for my organisation, date and scoring method 2 – System 4 – FAQ’s • Things to note: • Not all action types are available to all users. Which actions you have access to will depend on the user profile you have. • Within the majority of the My Analysis components there are links to the actions that you may wish to take at any given time.

1 – Background 3 – Tool My Resources and Administration • Purpose of My Resources: • The component provides access to a range of help resources for the system including: • FAQs • Help files • Training Guides • The section also provide access to the community forums / discussion groups • Purpose of Administration: • This component provides the SHA with the capability to control security access to information in the portal based on organisational preferences and reporting agreements • It also enables the SHA to manage the standardised list of workforce dimensions so that these can be updated should the need arise • As a user, you will not be able to view the Administer component of the portal 2 – System 4 – FAQ’s

1 – Background 3 – Tool Plan Data 2 – System 4 – FAQ’s Considerations: Uses year one data within the tool to provide the most robust and detailed information The 4 planning years are projections and therefore do not have the same degree of reliability Given the lead in time for education and training data that is four year ahead is needed to support education commissioning Understanding any quality and safety issues associated with the plans is important to ensure the most robust intelligence Workforce assurance tool needs to use the most accurate plans available and recognise that provider owned plans are often more complete for activity data as it reflects multi-commissioners activity levels

1 – Background 3 – Tool Freedom of Information (FOI) Requests 2 – System 4 – FAQ’s Considerations: The majority of the information used in the tool is already publicly available and therefore will remain eligible for FOI requests The planning data that will be within the tool is already held by SHAs and the DH. Access to the planning data will still only be available to the individual organisation, PCT Commissioning Cluster and SHA. The use of the data within the tool will be under the same FOI conditions as if requested from the SHA/ DH at present. The tool will not be used to publish data but will used to support workforce assurance which will include improving the quality and safety of workforce plans. Most SHAs already use Integrated Performance Dashboards which review plans and performance against plan, the only difference with this tool is the length of the plans reviewed.

1 – Background 3 – Tool Foundation Trusts data 2 – System 4 – FAQ’s Considerations: The range of data included in the tool for Foundation Trusts will be different to that of Non-Foundation Trusts. This issue is much wider than the workforce assurance tool and therefore it is proposed that the information available is used. The tool is not intended to collect additional information or intelligence and the project team will be reviewing the national discussions around quality, safety and Foundation Trusts to ensure it keeps up to date with any developments and continues to use the best available information.

1 – Background 3 – Tool Accessing the portal • Users access the portal by navigating to the Portal homepage in Internet Explorer • Users will be prompted to enter a username. • Navigate to the relevant URL: • Live Environment: https://wfa.westmidlands.nhs.uk • Log in using the username and password provided • Note that users are signed out of the portal after 30 minutes of inactivity 2 – System 4 – FAQ’s • Things to note: • An NHS N3 Connection is required to Access the portal • The portal operates over HTTPS to increase security • If problems occur the first ports of call are the Help sections in the My Resources tab of the tool, Training and pre-read material. • For further issues, speak to your local IT team or the Helpdesk – information on this is available in the about section of the portal (My Resources) • Helpdesk: Telephone: 0207 303 8777 • Email: wfp@deloitte.co.uk

Module 3: part 2 User Exercises To be completed

Module 4 FAQ’s

1 – Background 3 – Tool Questions: are Foundation Trusts included? • Will this be of benefit to Foundation Trusts? • Whilst having a differing status within the NHS, Foundation Trusts face comparable workforce pressures and a need to assure the appropriateness of their workforce. • It is envisaged that FTs will recognise the value that this tool can bring for a nominal investment and use it as a check and balance within their planning processes. • Will this be of benefit to Foundation Trusts? • At a system level, the assurance responsibility remains regardless of FT status, and proposals suggest that if anything this responsibility will be strengthened rather than reduced as a result of any changes under 'Liberating the NHS' • During their remaining period, there is an expectation that SHAs and PCT Commissioners / Clusters will 'hold the ring' on system assurance for all NHS services. • Do FTs have different data sets? • National data sources and public domain information will be utilised to minimise the information requirement on all NHS bodies. • Recognising that differing sets of data will be available for FTs, the system will use the information available to indicate risk. Obviously as the quantity and quality of information decreases so will the ability to effectively highlight risk areas and this will be clearly identified in the tool. 2 – System 4 – FAQ’s

1 – Background 3 – Tool Questions: is Primary Care included? • Primary care obviously has a large part to play in the assurance of a health system but it also marks the start of a large proportion of patient journeys. This importance is reflected within the assurance model by how we seek to measure referrals and supply of primary care workforce to try and judge the impact on the secondary sector. • Why are you not using QOF as a metric? • The lack of real variability in QOF scores makes using this data of limited benefit. • How are you linking primary care to the secondary care sector? • We are looking to link GP practices to secondary providers by mapping practices to commissioners and then identifying which providers these commissioners are “lead” commissioners for. • Why are you concentrating only on GP practice workforce and referrals? • We think these metrics should give a proxy for supply and coverage in the primary care workforce that can be matched with referral patterns, and subsequent secondary activity, to approximate demand emanating from primary care. We will look for correlations between these data points that can be used in system assurance. 2 – System 4 – FAQ’s

1 – Background 3 – Tool Questions: what is the assurance map The assurance map is a mechanism for considering potential workforce risks in a structured, logical and data driven manner. The map is hierarchical consisting of a number of categories and levels. 2 – System 4 – FAQ’s Assurance categories: There are four assurance categories that group together trigger groups that show: Assurance of plan (quality of the plan) Assurance against plan (variance analysis) Organisational workforce profile (general organisational indicators) Qualitative Indicators (not metric based but a prompt for areas to look for in an assurance process) Trigger groups: a collection of similar assurance triggers e.g. workforce or finance Triggers: a collection of similar metrics / indicators that can be seen as a natural group e.g. Staff Turnover or Cost Improvement Metrics: individual, national, local and bespoke metrics on workforce, quality, safety, finance, activity, etc. Assurance Categories High level Trigger groups Triggers Metrics Low level

1 – Background 3 – Tool Questions: how were the triggers decided The short-list of triggers were decided through consultation with regional reference groups / stakeholders within SHAs. We received a number of datasets from each SHA which had varying levels of detail, quality and confidence in the validity. It is recognised that metrics will be refined and further developed over time as more data becomes available, as reporting requirements change or further research shows that there are other factors or metrics that should be added or removed. NB. The triggers will be refined and added to as time goes on, and will be subject to ongoing clinical input and validation. 2 – System 4 – FAQ’s Looked at data available Looked at national recommendations and best practice Asked SHAs to review first cut of metrics and comment (using regional reference groups) • Refined metrics list on: • Availability of data; • Links to workforce, quality, safety or planning; and • Links to organisational health outcomes Ran a metrics workshop with key SHA representatives to agree core set of triggers/metrics Trigger group Triggers Metrics

1 – Background 3 – Tool Questions: how granular is the information? The initial metric set for the build will focus on overall organisation-level performance with some speciality/ward/condition level measurement. Where possible, within the constraints of the available data the statistical analysis will look for correlations at a more granular level. Where relationships are identified more granular metrics will be added to the tool. 2 – System 4 – FAQ’s • Activity • The majority of metrics will measure organisation level performance i.e. referral to treatment times, bed occupancy) • A smaller sub-set of metrics will look at speciality/condition level activity and performance i.e. LOS for select procedures, maternity performance • Finance • FIMS data will provide organisation level financial performance details i.e. total pay bill, clinical to non-clinical staff pay, % pay bill spent on overtime • Non FTs only • Quality/Safety • Quality and safety measures will exist largely at the organisation-level given the availability of data i.e. Number of SUIs, NPSA patient incidents • As more granular data becomes available it may be added to the system • Workforce • Workforce will look at staffing and overall workforce performance at the staff group level (Doctors, Nurses, Midwifes, Health Visitors)

1 – Background 3 – Tool Questions: how are data quality issues managed? Using statistical analysis to provide a data confidence and coverage score will ensure that the accuracy of information is incorporated into the assurance framework and system outputs. Creating a tool that users adopt will drive up data quality. 2 – System 4 – FAQ’s Definitions Confidence An automatically calculated measure of “data quality” and “predictability” of a data set Coverage An automatically calculated measure of “completeness” of the data underlying a metric score. This can be over time or within peer group Data Use Historic experience suggests that as assurers, commissioners and trusts realise the benefits of the system, the data will cleanse and the quality of national and plan data that is monitored will increase. Data Quality

1 – Background 3 – Tool Questions: what is the process of adjustment for data quality? Data is given a score representing confidence in the metrics which have been used to calculate it. Confidence is driven by the coverage and regularity of the data set. Each score is then weighted based upon its relative importance, for example recentness of gaps might be more significant than history length. The worst case scenario score for each indicator, used in a metric, is then taken. Scores are also time-dependent due to the history score and will be based on a rolling 3 year period. Factors that determine data quality are: 2 – System 4 – FAQ’s

1 – Background 3 – Tool Questions: who can see what information? When accessing the system users will be working with data for which they already have access (either because it is within the public domain, is their own data, or because it is information used in current assurance and/or performance management arrangements). The benefit of the system will not be enhanced access to data, rather enhanced access to data that has been coordinated, triangulated and presented in a way that optimises the workforce assurance process. 2 – System 4 – FAQ’s • Plans / scenarios can be “reviewed” in private by the Trust • Benchmarking by Trusts will be anonymous – you will see your position and the distribution, but not who the other data points relate to. • Ideally, the clinical value of the tool will encourage organisations to increase the role of Directors of Nursing, Medical Directors and Clinicians in decisions relating to workforce planning. Providers Commissioners Commissioners will be able to view data related to the providers from whom they commission. System Assurers Assurers will have a complete view of the data for those organisations they are responsible for assuring and not those of other regions/areas (nb. this may be flexed dependent on changes to the NCB regions in future)

1 – Background 3 – Tool Questions: why is data analytics important? • What additional capabilities will be enabled by data analytics? • The ability to use forecasting to help identify potential issues before they occur, and enabling a behavioural change in assurance – which should become mostly proactive rather than mostly reactive; • The ability to challenge accepted wisdom by supplementing expert opinion with robust statistical analysis; • The ability to accurately analyse trends and cumulative workforce changes to provide a more rounded picture of the potential impact these changes may have on quality and patient safety; • The ability to produce more sophisticated workforce metrics, and partly avoid the “yes, but it’s more complicated than that” factor; • The ability to present logical data-driven assumptions around workforce performance benchmarks and expectations; and • The ability to provide some level of “self-audit” to planners and provide feedback and challenge on their assertions around workforce plans and forecasts. 2 – System 4 – FAQ’s

1 – Background 3 – Tool Questions: what happens to the tool after the SHAs? Workforce assurance will take place in the proposed new structure for the NHS. With the challenges facing the NHS, There is a strong rationale for this being done by an overall assurer. This is, of course, to be decided and could be the remit of the National Commissioning Board, Health Education England or Local NHS Education and Training Boards 2 – System 4 – FAQ’s The tool will provide a robust way of assuring the workforce for commissioners and providers alike. The tool will provide Commissioning groups/Commissioning Clusters or GP Consortia confidence in current workforce performance and workforce plans, in relation to quality, safety and patient experience. It would also simplify negotiations around workforce plans and assurance which would condense timescales and potential tensions.

Workforce Assurance Tool Training Guidance Pack - DRAFT