EBA欧洲银行-Session-2-Slides-D.-Malikkidou2C20M.-BrC3A4uning_21页_1mb

Summary of "A New Approach to Early Warning Systems for Smaller European Banks"

Core Content

This document presents a new early warning system (LSI-EWS) specifically designed for smaller European banks, developed by the European Central Bank's Division Analysis and Methodological Support (DG Micro-prudential Supervision III). The system aims to enhance the ability of supervisors to identify potential distress in banks using a machine learning approach, complementing traditional expert-based methods.

Main Points

1. Motivation and Approach

• The LSI-EWS is developed to address the limitations of conventional early warning systems, which often rely on a small number of distress events and may lack transparency.
• A broadened definition of distress is used, including:
  • Breach of capital requirements
  • Early intervention measures under the Bank Recovery and Resolution Directive (BRRD)
  • Notifications from National Competent Authorities
• The system applies machine learning techniques, specifically a decision tree model, to improve predictive performance and transparency.
• The CRISP-DM methodology is followed to ensure a structured and robust development process.

2. Data and Pre-processing

• The study uses a unique dataset comprising over 3,000 small banks and approximately 350 distress events, covering the period from 2014Q4 to 2016Q1.
• Challenges identified include:
  • Data availability: Annual financial reporting with time gaps
  • Data quality: Missing values and potential errors
  • Data comparability: Differences in accounting standards (national GAAP vs. IFRS)
• To address these issues, data pre-processing steps are implemented:
  • Data cleaning: Removing banks and variables with insufficient data or high correlation
  • Data transformation: Adjusting for accounting differences and creating ratios (e.g., RoA, RoE, NPL ratio)
  • Variable selection: Ranking variables by predictive importance (AUC) and selecting the top 20 for the final model, complemented by expert judgment

3. Model Development

• A decision tree model is constructed using Quinlan's C5.0 classifier with boosting to improve accuracy.
• The model is built to predict distress within the next quarter, with a prediction horizon from 2015Q1 to account for publication lags.
• A conservative approach is used during modeling, where the cost of misclassifying a distress event is assumed to be twice that of a non-distress event.
• The final model includes 19 nodes and 12 explanatory variables, with profitability as the primary indicator.

4. Model Validation

• The model is validated using in-sample and out-of-sample testing (75% vs. 25% split).
• Performance is measured using:
  • AUC (Area Under the Curve): 0.95 (in-sample), 0.92 (out-of-sample)
  • Cohen's Kappa: 0.886 (in-sample), 0.803 (out-of-sample)
• The decision tree model outperforms the logit model in terms of capturing distress events.
• Type I and Type II error rates are also reported:
  • Type I error rate: 0.010 (in-sample), 0.033 (out-of-sample)
  • Type II error rate: 0.109 (in-sample), 0.099 (out-of-sample)

5. Conclusion and Next Steps

• The LSI-EWS provides a forward-looking and transparent tool for identifying bank distress, which is critical for effective supervision.
• The system is promising based on initial results and further backtesting.
• Future improvements include:
  • Enriching data with FINREP and additional variables
  • Extending the prediction horizon to up to six months
  • Differentiating between severeness of distress events
  • Deploying the system in the supervisory process

Key Information

• Target Variable: A bank is in distress if it meets any of the following:
  • Conventional distress (bankruptcy, liquidation)
  • Early intervention under BRRD (capital thresholds breach)
  • Special administration or deemed to be failing
  • Rapid financial deterioration
• Modeling Method: Supervised learning using a decision tree classifier with boosting.
• Validation Metrics:
  • AUC: 0.95 (in-sample), 0.92 (out-of-sample)
  • Cohen's Kappa: 0.886 (in-sample), 0.803 (out-of-sample)
• Variable Selection: Top 20 variables based on predictive importance, with expert judgment used to finalize the model.
• Data Challenges: Addressed through in-depth data preparation, including normalization and accounting adjustments.

Annex: Validation Terminology

• TP (True Positive): Correctly identified distress events
• TN (True Negative): Correctly identified non-distress events
• FP (False Positive): Non-distress events incorrectly flagged as distress
• FN (False Negative): Distress events incorrectly not flagged
• AUC: Measures the overall performance of the model, with values between 0.9 and 1.0 indicating excellent performance.
• ROC Space: Represents the trade-off between sensitivity (TPR) and specificity (1 - FPR), with the shadow area indicating better classification.