KRIs – a failed experiment or a misunderstood tool?

1. KRIs – a failed experiment or a misunderstood tool? Scottish Chapter Annual Conference 1 November 2013

2. Disclaimer • Your speaker is a Fellow of the Institute of Directors, a Director of Council and Vice-Chairperson of the IOR, as well as the Chief Executive of RiskBusiness International Limited. • The views expressed in this presentation are the sole responsibility of the presenter and do not and can not be construed as representing the view of either the Institute of Operational Risk or RiskBusiness International Limited.

3. What is a KRI? • Basically, whatever you want it to be! • It’s a metric, a piece of information • It only has value to those who can use it • There is no difference between KRIs, KPIs and KCIs – depends on source and use of the information

4. Why is Op Risk “Hard” to do? • It’s fundamentally different to other types of risk: • No direct correlation to volumes, market volatility, economic cycles or other easily quantifiable factors • Its embedded into each of the other types of risk – how and when do we separate them? • It has a direct link to the “human factor” • The business intuitively accept it as part of “business as usual” and have difficulty in understanding the “regulatory” rationale behind elevating it to a distinct risk class • The issues of “face”, deniability, ignorance, the “Titanic” effect and historical audit processes • Multiple independent programs

5. Why are KRIs left to last? • How do we correlate them to…… • Risk? • Exposure? • Loss Data? • Capital Requirements? • Senior Management Requirements and General Business Management Needs? • Where do we start? • Can they ever be predictive? If not, why bother?

6. Predictive metrics • By themselves, individual indicators will never be truly predictive • Avoid thinking that trends are predictive • KRIs can be used to “predict” events before they manifest themselves – if: • There is a common structure against which risks and losses are classified and to which indicators are linked • Indicators are monitored on a real-time basis by business units carrying the exposure • Management reacts to stimuli as and when warning patterns emerge • We have had sufficient history to test and prove the correlations

7. The role of KRIs in time

8. Who owns KRIs? • Who decides which KRIs to monitor? • Who decides how the KRI values are calculated? • Who decides who receives KRI reports and when they receive them? • Who decides when to escalate a KRI submission? • Who decides to discontinue using a specific KRI?

9. Roles in the KRI Process Providers Producers Risk Management Observers Consumers

10. Operational Risk as a “Bow Tie” Events Causes Impacts

11. The Role of KRIs Internal causes EVENT External causes Detective Controls Preventive Controls Expected Result Defined Business Process Defined Business Process Risk Category Oversight Controls KRIs

12. However.... • The Bow-Tie Model suggests that causal factors drive everything else, therefore the normal business model needs to in corporate cause as a major component

13. Causal chains Manager Annual holidays S’visor A S’visor B Clerk A Clerk A Clerk A Clerk B Clerk B Clerk B Short staffed Staff personal problems Lack of training Market volatility Correct? Send to Confirm Trade Record Review and check details Yes Complex trade Staff € problems M’ment offsite No System failure Return to F/O

14. Evolution of risk classification Internal causes Risk External causes Business Process Preventive Controls Preventive Controls Process Risks Process Risk Causes Detective Controls Detective Controls Risk Category

15. Data Management • Quality management is one of the most critical aspects of the KRI programme: • Use standard specifications covering what, when and how data will be measured, along with who and why • Investigate discrepancies and challenges • Utilise other LoD to assist in QM • Collect data at the earliest opportunity, disseminate it as quickly as you can • Consumers often already have the data, add value by augmenting the data with context • #/$ of suspense account items • Broken down by time-buckets • Analysed by party/system/entity

16. Top-Down v Bottom-Up • The concept is to obtain reports on the same metrics from across the firm • Can we identify any metrics which can actually be compared across the entire firm? • Are these metrics in any way correlated to risk exposure?

17. Derived indicators • “Manufactured” metrics whose value is derived from a series of underlying metrics • Examples of underlying metrics: • Staff availability • Average error count • Average cost per error • Transactional volumes • Economic cycles • Cycle volatility • The resultant metric seeks to measure the impact of staff taking leave at specific points in time, assessing the increased potential cost

18. Thresholds • Thresholds are essentially pre-defined limits which, when the value of a KRI reaches that level, generates warnings or alerts • Different types: • Touch • Repetitive touch • Percentile breach • Trend threshold • Thresholds should never be absolute; they need to address cycles and correlations.

19. Threshold issues • Never start with a “big bang” – management will be confused if they suddenly start receiving numerous “alerts” and “warnings” • Start with high-level thresholds and fine tune them over time – what is the correct level for any metric? • Use layered structures so that increasing levels of seniority receive warnings when appropriate – and the “worker-bees” do not get a surprise • Revise thresholds from time to time

20. In summary…… • There is no such thing as a global set of “top 10” indicators which everyone should monitor • Excluding composite or index-based metrics, indicators are not in of themselves predictive • A good indicator programme will involve a large number of players • Your risk profile is constantly changing, causal drivers continuously morph into different impact chains – the indicator set being monitored should thus not be set in stone • Don’t expect instant gratification – the benefits of the indicator programme will take time to manifest themselves • Read the IOR SPG on KRIs issued November 2010!

21. Questions and Comments • Contact Details: • Mike Finlay, Chief Executive, RiskBusiness • Telephone : +44 7721 969 224 • E-mail : mike.finlay@riskbusiness.com or mfinlay@ior-institute.org • URL : www.riskbusiness.com and www.ior-institute.org