IOPS Toolkit for Risk-based Supervision

1. IOPS Toolkit for Risk-based Supervision Module 2: Quantitative Assessment of Risk

2. Overview • Quantitative assessments can play an important part RBS - poor results from quantitative tests imply higher levels of residual risk is then factor into its overall risk analysis or risk score • Quantitative tools can provide a bridge between compliance and risk-based approach to supervision • Quantitative tests can be undertaken by the supervisory authority itself or by pension funds • Quantitative risk assessessment should focus on the risks relevant for the type of fund (i.e. funding and solvency issues for DB funds, investment returns and volatility and reliability of retirement income for DC funds) • Not all risks can be measured in a quantitative fashion – the overall risk assessment will always be a combination of quantitative and qualitative factors

3. Models Quantitative assessment tools make use of models – 3 main problems: • what is modelled and how: including omitting factors which turn out to be vital, the use of inappropriate data, only using recent data in the development of models and their parameters, with models failing to take account of extreme events and modellers all tending to use the same accepted view • understanding the power and limitations of models: including their use outside their sphere of applicability, hidden assumptions and simply overestimating the power of models which are by their very nature simplified representations of the real world • operational risks around the use of models: such as poor documentation, lack of testing, and the misuse of data

4. Models Recommendations include that models should: • sufficiently represent those aspects of the real world that are relevant to the decision makers for which the information will be used • include explanations of how the inputs are derived and what the outputs are intended to represent • be fit for purpose in both theory and practice • include explanations of their significant limitations

5. Quantitative Regulations • Quantitative regulations are the starting point for quantitative risk analysis • These can be straight forward limits (such as minimum funding rules for DB funds or investment restrictions for DC funds) • Alternatively, these quantitative regulations can be risk-based themselves (e.g. factor based solvency rules, VaR calcuations) • Under a risk-based approach, supervisors need to consider not only whether quantitative regulatory requirements (such as solvency stress and scenario tests and value at risk (VaR) measures) are being met, but whether risks are being identified and managed in such a way that the requirements will continue to be met in the future

6. Quantitative Regulations • Risk-based supervision can incorporate quantitative regulations in 3 ways: • The most simple fashion is to combine a ‘compliance’ and a ‘risk-based’ approach – compliance with quantitative restrictions is checked, and if not met a lower score would be factored into the overall risk assessment of the fund • Alternatively, these straight forward quantitative requirements could be made more ‘risk-based’ but testing whether compliance would still hold in adverse circumstances (i.e. by stress testing) -the results of these stress-tests would then be incorporated in the overall risk score • Where the quantitative regulations are already risk-based compliance with these risk-based regulations would be fed into the overall risk score

7. Quantitative Regulations: DB Funds • Valuation requirements : these can be tested and incorporated into an overall risk analysis by looking at the valuation assumptions, undertaking sensitivity testing of changes in valuation assumptions, and stress testing of risks such as high inflation • Minimum funding requirements: these can be incorporated into an overall risk assessment either by checking for compliance (and scoring the fund accordingly) or by stress testing the funding position to see if the minimum requirements would be met under adverse circumstances (with the results of such tests fed into the risk score) • Factor-based solvency margins: solvency margins can be risk-based by requiring higher amounts of capital to be held against risky assets (such as equities), thereby providing a buffer in case such assets decline in value. Again either straight forward or stress-tested compliance with these margins can be incorporated into an overall risk score • Stress-related solvency margins : require each entity to calculate the additional amount of assets it would need to be able to meet its obligations under a prescribed stress scenario or scenarios. The results are then fed into the overall risk assessment

9. Quantitative Regulations: DC Funds • Investment limits: compliance with these limits forms part of the overall risk score • Minimum return requirements: solvency requirements backing these guarantees would be measured and assessed in the same way as DB fund promises • Value at risk limits: these assess the volatility of investment returns . Results of such stress testing would be incorporated in the overall risk score. Where such limits are themselves regulatory requirements, compliance would be part of the overall risk assessment • Alternative risk measures: attempt to measure risk measurements against long-term income requirements (such as replacement ratios), with regulators devising optimal portfolios for achieving this target. The performance of the actual portfolio of a pension fund could then be assessed vs. this benchmark portfolio. Supervisors could then work this analysis into their overall risk assessment via a ‘traffic light’ system - for example a green light would indicate a pension fund with a portfolio structure aligned with the benchmark and a good risk management system.

11. Techniques for Quantitative Risk Assessment • comparison of valuation assumptions – compare assumptions with peers to identify if inappropriate • analysis of surplus –compare actual experience to assumptions to see if ones using are accurate / appropriate • roll-forward calculations –financial position projected under certain scenarios to assess exposure to adverse circumstances • duration analysis – project cash flows of assets and liabilities of fund and compare interest rate sensitivity and timing mismatches • sensitivity testing – test sensitivity of valuation results to different assumptions by recalculating results using different assumptions • deterministic stress testing – calculate the financial position of a pension entity at current or future date to 1 or more adverse scenarios • stochastic stress testing –adverse scenarios computer generated not predefined and distribution of results examined (i.e. likelihood that scenarios adverse enough to create financial difficulty will occur) • value at risk (VaR) calculations – type of stochastic stress test measuring adverse market movement with a specified probability

12. Techniques for Quantitative Risk Assessment

13. Integrating Quantitative Tools DB • Solvency + funding ratios are the key quantitative tests for DB funds – above 100% resulting in low risk score, below implying a higher level • Roll forward + stress tests would tests these further – if funding remains over 100% after stress testing a low risk score would result • ALM testing (matching assets and cash flows to liabilities) could also be taken as a sign of robust risk management at a pension fund

14. Integrating Quantitative Tools DC • VaR – measuring investment volatility can be used for DC funds, though is controversial • ALM type measurements to see if DC funds can meet income replacement targets are still under development • For DC funds offering guarantees, similar standard and solvency stress tests to DB funds can be used

15. Quantitative Measurement of Non-financial Risk Ratings varying between 0-1 could be given for risks such as: • DB plans having a number of complicating features, such as early retirement benefits, indexation etc. • DC plans offering a large range of investment options, rather than having just a few investment funds or “life-cycle”; having one fund for all, but not allocating investment earnings on a market basis, but “declaring” the rate on a non-transparent smoothing approach and building up “reserves” ; high levels of outsourcing.