1 / 15

Discussant: Chen Zhou De Nederlandsche Bank

Discussion on “Diagnosing the Financial System: Financial Conditions and Financial Stress” by Scott Brave and R. Andrew Butters. Discussant: Chen Zhou De Nederlandsche Bank Views expressed do not necessarily reflect the view of DNB. The paper. An indicator of financial condition (NFCI)

josephhunt
Download Presentation

Discussant: Chen Zhou De Nederlandsche Bank

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Discussion on “Diagnosing the Financial System: Financial Conditions and Financial Stress” by Scott Brave and R. Andrew Butters Discussant: Chen Zhou De Nederlandsche Bank Views expressed do not necessarily reflect the view of DNB

  2. The paper • An indicator of financial condition (NFCI) • Thermometer for financial system • Based on a large scope of macro/finance data • Aggregating data with different frequency • NFCI as an indicator/predictor for financial stress • Normal v.s. stress (crisis threshold) • ROC curve in threshold selection

  3. An example in diagnosis • Diagnosing a rare disease • Very rare disease Pr(Virus)=0.01% • Very reliable doctor/method • P(Positive|Virus)=99.9% • P(Negatie|No Virus)=99.9% • When getting a positive test result, should the patient worry? Calculating Pr(Virus|Positive)

  4. Calculation… • Bayes formula The patient does not have to worry too much!

  5. What went wrong with the diagnosis? • Why Pr(Virus|Positive) is not high • The diagnosis method seems to be reliable: Pr(Positive|Virus) and Pr(Negative|No Virus) are both very high • However, Pr(Virus|Positive)≠Pr(Positive|Virus) • If Pr(Positive)>P(Virus) Pr(Virus|Positive)<Pr(Positive|Virus) • If a positive signal occurs too often, then it is less informative.

  6. A Close look at the formula • As Pr(Virus) tends to zero, Pr(Virus|Positive) can be very low. Hence it is very difficult to diagnose rare disease. Very high 999 9999 Should be very high to offset the effect that the disease is rare

  7. Lessons learned • Diagnosing rare disease • Having low Type I and II errors is not sufficient • It is necessary to have a diagnosis system that do no produce many “positive” signals. • It is necessary to have Type II error much lower than Type I error Rare disease is very difficult to diagnose by nature!

  8. Back to diagnosing the financial system • How about the NFCI as an indicator? • Financial stresses/crises are rare: • In this paper P(Crisis)=52/1983=2.62% • With the threshold at -0.37, from ROC • Pr(Signal|Crisis)=91% • Pr(Signal|No Crisis)=19% • Thus Pr(Crisis|Signal)=11.4%, quite low! • Reason: Pr(Signal)=20.9%>>Pr(Crisis) • See also Figure 3

  9. How to improve? • To diagnose rare events, it is necessary to have Type II error much lower than Type I error • Currently: Type II error: 19%, Type I error 9% • Along the ROC curve, can we improve? Here? Type II: 2% Type I: 38%

  10. At the new point • Calculations: • P(Crisis)=52/1983=2.62% • Pr(Signal|Crisis)=62% • Pr(Signal|No Crisis)=2% • Thus Pr(Crisis|Signal)=45.5% • Better, but still not satisfactory • Threshold: • Much higher than -0.37 at the new point!

  11. Which conditional probability? • Comparison • In current paper: Pr(Signal|Crisis) and Pr(No Signal|No Crisis) • We talked about: Pr(Crisis|Signal) and Pr(No Crisis|No Signal) • Why the latter is more important than the former? • Utility function: decisions from the signal

  12. Can we improve? • Absolute performance • Difficult to have Pr(Signal|Crisis) and Pr(No Signal|No Crisis) both at high level • Not only NFCI, but also other indicators • Difficult as it is • Relative performance • Can still be compared • Utility function based on the new sets of conditional probability

  13. An alternative ROC curve • Current ROC curve based on Pr(Signal|No Crisis) v.s. Pr(Signal|Crisis) • New ROC curve based on Pr(Crisis|No Signal) v.s. Pr(Crisis|Signal) • All existing techniques such as AUROC are still valid for examining the relative performance between the NFCI and other indicators • The choice of threshold: based on the new utility function and the new ROC curve

  14. Not yet about forecasting • Indicating crisis is already so difficult! • Forecasting is even more! • Alternative measure on forecasting performance • Current: a “hit rate” measure • Alternative: Probability forecasting? • NFCI might be linked to a good probability forecast of the distress • Evaluating probability forecast Always difficult to handle rare events!

  15. Two main messages When working on indicator/predictor of financial crisis (or any rare event) • Should look at the right measures for evaluating the performance • Should not be disappointed when looking at them

More Related