Current challenges in supervision – new challenges for researchers

Keynote speech by Claudia Buch, Chair of the Supervisory Board of the ECB, at the 2024 Annual ECB Banking Supervision Research Conference

Frankfurt am Main, 11 June 2024

After the 2008 financial crisis, some called for a return to “boring banking”.^[1] More than 15 years later, banking remains anything but boring. Strong governance and risk control mechanisms are needed to deal with emerging risks. Strengthening and maintaining resilience in turbulent times requires adequate capitalisation, sufficient liquidity and a high degree of operational resilience.

Similarly, banking supervision remains far from dull. Turmoil on international banking markets in March last year showed that fragilities in banks’ risk management and governance can become exposed if adverse shocks hit. If supervisors fail to follow up on their findings quickly and effectively, they may have to manage rapidly emerging crises. This is despite the significant progress that has been made in increasing banks’ resilience and strengthening resolution frameworks.

So, supervision is challenging. Supervisors need a sound understanding of idiosyncratic, bank-specific risks. They need to understand the macroeconomic and competitive environment in which banks operate. They need to assess how future events, which are unpredictable and highly uncertain, might affect the sustainability of banks’ business models and risk profiles. Cyber risks, climate-related and environmental risks or geopolitical risks are not entirely new – but their intensity has increased and there is little recent historical experience of these risks materialising.

You could call it a risk assessment paradox. Precisely at a time when forward-looking risk assessments are needed to cut through the uncertainty that surrounds us, existing models may not be very useful and analytical tools are often not very sharp.

These are the challenges that supervisors need to respond to. They need the right skills and mindsets to assess and address risks in banking, and they are restricted by the available resources. In this context, the Supervisory Board of the ECB has just decided on a significant reform to make European banking supervision more effective and efficient.^[2]

So where does the perspective of researchers come in? Having spent many years doing banking research myself, I am a firm believer in the power of state-of-the-art analytical work in providing clear, data-driven insights that can inform decision-making.

Today I would like to discuss two main areas where analytical tools come into play: assessing the effectiveness of supervision and improving forward-looking risk assessments.

How can we assess supervisory effectiveness?

Evaluating supervisory effectiveness requires a structured process.^[3] It starts with the strategic objectives of supervision, which need to be translated into concrete, measurable indicators. These indicators must capture both intended and potential unintended consequences of supervision, and the relevant data must be available in a timely and consistent manner. Then there are different measures supervisors can take and instruments they can use to make sure strategic objectives are being met. Research can support this process by providing conceptual frameworks to analyse how supervisory measures can affect outcomes – both intended and unintended.

The strategic objectives of supervision are very clear. In essence, supervision is about identifying risks in banks and getting them to remediate weaknesses in a timely manner – all to ensure that banks are sufficiently resilient.

In terms of indicators, the ECB’s Supervisory Review and Evaluation Process (SREP) focuses on four main areas.

First, banks’ capitalisation is the key metric for assessing resilience. Whether banks are sufficiently capitalised is determined by assessing their vulnerability to four risk categories: credit risk, market risk, interest rate risk and operational risk. This risk assessment needs to be done in a forward-looking way – an aspect I will return to later on.

The second crucial component of resilience is liquidity. This applies particularly in the current environment of quantitative tightening, reduced liquidity provision by central banks, and interconnectedness between banks and non-bank financial intermediaries, which may be highly leveraged.

Third, beyond capital and liquidity, the viability of banks’ business models is assessed to understand whether their ability to generate returns is sustainable in the short, medium and long term.

Finally, banks’ risk management and internal governance are evaluated with a view to assessing their overall organisational competence and capacity. This is done by examining how a bank is run, including its key decision-making and organisational structures, its risk appetite framework as well as risk culture, including remuneration policies. Supervisors also look at the infrastructure that is used to aggregate and assess relevant information.

Assessments in these four categories lead to SREP scores – one for each category, and one overall score. These scores are the basis for setting quantitative capital requirements.

Once deficiencies are pinpointed, supervisory measures are deployed to address and rectify them. For instance, if capital adequacy is found lacking, supervisors may impose higher capital requirements. When compliance with those requirements is endangered, supervisors may demand a capital restoration plan. If liquidity is insufficient, banks are required to maintain higher liquidity buffers and improve their liquidity risk management practices. To address weaknesses in the business model, supervisors can mandate strategic adjustments. In cases of deficiencies in internal governance and risk management, actions focus on improving the management body, the risk management function or internal audit. These tailored supervisory actions aim to rectify identified deficiencies, ensuring that the banks meet prudential requirements^[4] and have sound management and coverage of risk.^[5]

Supervisory measures differ in terms of their effects, their intrusiveness as well as the resources they require. Increasingly, European banking supervision is embedding supervisory actions in an escalation ladder setting out a time-bound remediation path. This gives banks the opportunity to address the root cause of a problem within a defined timeline. And if a bank fails to fix the issue, supervisors can follow up with supervisory tools of various levels of intrusiveness. These range from moral suasion and letters to legally binding decisions, which, if needed, supervisors can enforce through monetary penalties, such as periodic penalty payments and sanctions ensuring compliance with prudential requirements and deterring future misconduct.^[6]

But how do we know whether the measures are effective in getting banks to comply with supervisory standards?

Good supervision certainly involves both good judgement and good analytics. Daily work of banking supervision produces a wealth of information – on banks’ performance, on supervisory measures and instruments, on banks’ risk scores. Applying analytical tools to structure this information can help assess the effectiveness of supervisory measures, providing solid ground for the exercise of judgement.

The path from supervisory actions to desired outcomes is often winding. For the individual Joint Supervisory Team, it is often difficult to assess whether the action taken has the intended effects. If a specific action has been requested by supervisors and is then taken, causality and effectiveness are relatively clear. In reality, though, many external factors influence the performance of banks and many internal risk drivers are relevant.

Indeed, measuring supervisory effectiveness faces several challenges.^[7] First, establishing causality between supervisory interventions and bank-specific outcomes is not easy. Changes in a bank’s risk profile might result from external economic conditions beyond supervisory actions. Natural experiments and sound counterfactual analyses are rare. Second, there can be time lags between supervisory actions and their outcomes. Supervisory measures might have a negative impact on a bank's immediate financial position but reduce long-term failure risk. Third, focusing on quantitative performance metrics has its pitfalls, as it may divert attention from more qualitative but critical activities.

These challenges are not new and they beleaguer any impact assessment or policy evaluation. And there are indeed many solutions and analytical tools available to help navigate these challenges. Many research studies have in fact successfully analysed the effectiveness of supervisory measures. Let me give you a few examples:

• One set of studies deals with the intensity and types of supervisory action. This research shows that better-rated banks receive more supervisory attention measured as the number of hours spent by supervisors. This seems to pay off in terms of better asset quality, less volatility, and less sensitivity to industry downturns, without weakening growth or profitability.^[8]
• On-site inspections are effective as well, leading to more conservative risk management in banks. But these inspections also need to be sufficiently frequent to have relevant effects.^[9]
• Stress tests lead to decreased risk levels for banks.^[10] Increased supervisory scrutiny during the exercise amplifies the effect.^[11] The publication of results improves market discipline and financial stability.
• The powers to change banks’ managers and organisational set-up reduce riskiness, unlike monetary penalties.^[12]
• Finally, dividend recommendations during the pandemic crisis supported lending without increasing it for riskier borrowers or “zombie firms”.^[13]

Sound analytical work thus provides evidence-based insights for identifying the most effective supervisory actions. And I am delighted to see that a number of the papers being presented at today’s conference go exactly in this direction.^[14]

But a structured approach to assessing supervisory effectiveness can be improved by better aligning the incentives of researchers and supervisors. To be useful for supervisory decision-making, studies should not only cover specific countries or episodes, but also replicate findings across different environments. However, such replication studies often go unrewarded in the research community. Also, supervisors have to deal with many acute challenges every day and often lack the capacity to conduct structured impact assessments.

One way of integrating assessments into routine supervisory work is through “second line of defence” functions which evaluate the consistency and effectiveness of supervisory measures. ECB Banking Supervision established its second line of defence in 2020. This function compares supervisory decisions and bank performance across the more than 100 significant institutions directly supervised by the ECB. Such benchmarking allows supervisors to assess the relative performance of banks in a way that they could not do solely at national level as relevant peer groups may not exist. At the same time, the second line function ensures that supervisors can sufficiently harmonise their action across different banks.

Last but not least, evaluating supervisory effectiveness requires a robust infrastructure, a core element of which relates to data. European banking supervision’s suptech strategy is central to this data infrastructure as it provides tools for readily accessing and processing data. In addition, supervisors need to access evaluation frameworks and past evaluations through repositories of relevant studies. A promising template is the BIS’s Financial Regulation Assessment: Meta Exercise, FRAME, which is an online repository of studies on the economic impact of various types of financial regulations.^[15] Impact assessments and evaluation frameworks related to banking supervision could be collected in a similar way.^[16] As regards incentives for researchers, a better acknowledgment of replication studies in the academic community would certainly be helpful.

How can we address risks in a forward-looking way?

The second area where supervision and research can fruitfully interact is in identifying risks in a forward-looking way.

The risk environment in which banks operate has evolved substantially in recent years. Structural shifts and external shocks have made risk assessments more complex. The risks facing banks are affected by geopolitical risks, changes in supply chains, inflation, macroeconomic uncertainty, climate change, nature degradation and digitalisation. And this may not even be an exhaustive list of such novel risks.

What is novel about these risks? Risks are novel if they have not materialised over the past years, implying that banks’ risk assessment models lack sufficient information and data to account for them. Take climate-related risks: climate change has been a known phenomenon for a long time. Yet the related transitional and physical risks materialise only over a long time horizon and do not affect all regions equally. Banks hence need to develop new tools and procedures for assessing and managing these risks.^[17]

Similarly, while geopolitical risks are certainly not new, they had been subdued for a long time and this needs to be reflected in banks‘ risk management. Geopolitical tensions can lead to economic sanctions, trade restrictions and disruptions in international relations, all of which can negatively affect borrowers' ability to repay loans and so increase credit risk. Cyber risks are exacerbated by geopolitical conflicts.^[18] Moreover, geopolitical instability can disrupt the payments system. Cross-border payment channels may be affected by sanctions or cyber attacks, leading to delays or failures in transactions.

What’s more, geopolitical developments not only create risks, but fundamental uncertainty. Risks can be quantified and managed through traditional risk assessment tools and models. Banks can estimate the likelihood of loan defaults based on historical data and develop strategies to mitigate these risks accordingly. However, uncertainties involve variables that cannot be easily quantified or forecasted.

This new risk environment poses a significant challenge as we cannot meaningfully draw on historical data series and models that have previously worked well. Proxies of credit risk, for example, will not properly capture climate-related risks if no adverse climate-related events have materialised in the past. Credit risk models may not properly estimate nor forecast the effects of adverse shocks as, during the most recent deep recession related to the pandemic, credit losses were largely contained thanks to fiscal policy support. That’s why banks need alternative approaches to quantify and cover novel risks.

This problem needs to be addressed by banks’ risk management. Many banks use overlays for adjusting their expected loan loss provisioning to reflect novel risks.^[19] Overlays in IFRS 9 provisioning models account for emerging risks and uncertainties. They are applied on top of model outputs to cover potential losses from risks not adequately captured by the models.

Because overlays have a key impact on bank resilience, the ECB has surveyed current practices. Overlays became prevalent during the pandemic and are now used for different novel risks not easily captured by models, such as energy supply, supply chains in general, environmental risks, inflation and geopolitical risks. While progress has been made, especially in the area of climate and environmental risks, many banks are still far from meeting the expectations of IFRS 9.

Some banks that use so-called “umbrella overlays”, which fail to account for sectoral differences in case a risk driver has a different impact on borrower groups. Also, some banks just lower the general GDP forecasts that feed their expected loss models, but this ignores that for example a disruption of trade patterns may threaten some export-oriented clients while it might only marginally affect aggregate GDP. This practice underestimates the true default risks that banks are facing. And while banks are right in using overlays to calculate their expected credit loss, many ignore that the same risks should also trigger a stage transfer from performing to underperforming clients. This, too, would lead to an under-provisioning for novel risks. Hence, good risk management in banks requires improving the use of overlays to consider the impact of novel risks more precisely, to use simulations and scenarios, and to improve stage transfers.

Supervisors face the same fundamental uncertainty in assessing future risks. Combining macro and micro perspectives in an iterative way can provide useful insights. Central banks are well positioned to do this, given their dual mandates for micro- and macroprudential oversight and access to both aggregate and granular data. Central banks have a broad oversight of the economy and different sectors. Macro analysts can contribute by drawing on historical time series of similar risk episodes. From the supervisory side, central banks also have information on individual banks.

What infrastructure and tools do we need for this? First, we need common data. A good example is AnaCredit, short for Analytical Credit Dataset, which is a comprehensive database developed by the ECB to provide detailed information on individual bank loans within the euro area.^[20] It collects granular data on credit exposures and loan characteristics, helping to improve risk assessment and monitoring in the banking sector. In the future, the potential of this database could be further enhanced through feedback loops. Such feedback loops are available for some credit registries at the national level and provide semi-aggregate information to reporting banks.^[21]

To effectively combine macro and micro perspectives, we need good analytical tools. These tools facilitate the integration of micro-level information, such as individual loan data, and macro-level trends, like GDP growth and inflation rates. For example, Factor-Augmented Vector Autoregressions (FAVAR) models are empirical tools designed to incorporate a large set of macroeconomic and microeconomic data, capturing a wide array of economic dynamics.^[22]FAVAR models can provide insights into how changes in broad economic conditions might affect specific sectors or entities, and vice versa. This dual perspective is crucial for helping central banks and supervisors anticipate and respond to financial instability and other economic shocks. But alternative modelling approaches exist as well, and having different models in the toolbox is indeed useful in checking the robustness of findings.^[23]

In addition, model repositories serve as centralised databases for storing validated models and analytical methods and making them accessible to stakeholders. These repositories ensure that best practices are shared and that the models used are rigorously tested and validated. Shared repositories reduce the duplication of effort in model development and validation. Additionally, model repositories promote transparency by providing a clear view of the methodologies and assumptions underlying various analyses.

Better cooperation to master current challenges

Supervisors and banking researchers share many similarities. Supervisors work in the public interest and researchers seek to contribute to the public good. Both need critical thinking to challenge popular narratives and identify the core drivers and effects of risks in banking. Both need good analytics to do so.

But supervisors and researchers also have different incentives. Supervisory decisions often have to be taken under considerable time pressure. And supervisors need to focus on individual banks. By contrast, researchers can analyse patterns across many banks and over a longer time frame. They are incentivised to generate novel results and causal relationships, whereas supervisors need to embed impact assessments in routine processes.

However, there are many ways in which supervisors and researchers can fruitfully cooperate to master today’s challenges. Supervisors need to assess the effectiveness and efficiency of their actions; researchers can develop analytical tools that help them to do so. Supervisors need tools to assess future risks in a forward-looking manner, combining bank-level and aggregate information in intelligent ways; researchers can help by developing the necessary tools.

But to make this cooperation happen, we need to invest more in the infrastructure. This includes setting up repositories of replication studies and models, developing evaluation frameworks and further improving the data infrastructure. And even more importantly, we need to identify relevant questions through a continuous dialogue. Here, the critical mindset of researchers can help us in identifying issues that we generally do not talk enough about.^[24]