Challenges in Validation: Taking the Study Findings Forward A Corporate Perspective

1. Challenges in Validation: Taking the Study Findings ForwardA Corporate Perspective Advanced IRB Forum New York, June 19, 2003 Lyn McGowan RBC Financial Group

2. The Challenge of Validation for Corporate and Mid-Market Portfolios • Internal rating validation approaches, methods, issues vary, depending on the types of rating models used • Rating system design and validation go hand in hand Type of Rating ModelCORPORATEMID-MARKET Statistical Models 7 4 External Vendor Models 7 2 Expert Judgement Models 15 11 Hybrid Models 10 7

3. The Data Challenge • Insufficient data to rely on purely statistical means of validation  must rely on other means • The Basel Research Task Force recognizes that quantitative statistical techniques should be performed, however should not drive the pass/fail decision for IRB validation • Supervisor will need to understand and be satisfied with: • The logic of the risk assessment process • The rating system’s design and operation • How the rating system has been calibrated • The internal validation process • The “feedback loop”

4. Logic of the Risk Assessment Process • Conceptual clarity  Well-defined drivers/factors  Dynamic properties, significance of factors • Transparency Explicitly demonstrates reasoning  Constraints (such as stipulated factor weightings)  Assessment horizon • Replicability  “Gut feel” won’t do  Criteria or thresholds for factors • Well-documented  Process/procedures manual

5. Rating System Design and Operation • Conceptual clarity  Understandable output • Transparency  Not a Black Box • Replicability  Well-defined framework and/or methodology • Consistency  Application across industry, geography • Documentation  Rationale for design  Conceptual meaning, definition of grades  Frequency of review  Override authority, reporting

6. Calibration of the Rating System • Conceptual clarity  Techniques have been combined rationally • Transparency  Availability of data across quality spectrum  Method of mitigating scarcity of data  Basis for numerator and denominator • Consistency  Potential sources of bias  Relevance of external data • Documentation  Specific techniques used

7. Internal Validation Process • Conceptual clarity  Discriminative power vs accuracy of calibration  Factor relevance vs factor weights vs model strength  Rationale for Triangulation • Transparency  Scope / frequency of work  Mapping processes  Objective metrics • Consistency  Actual vs predicted  Own to external loss experience  Role of loan review unit • Documentation  Clear, comprehensive, precise

8. The Critical Feedback Loop Calibration Continuous Improvement Cycle Validation