The future of stress testing – some further thoughts
Speech by Andrea Enria, Chair of the Supervisory Board of the ECB, 8th Annual Research Workshop “The future of stress tests in the banking sector – approaches, governance and methodologies”, organised by EBA
Paris, 27 November 2019
Since the financial crisis, stress tests have become an important tool for banking supervisors. But they are also a complex tool. The underlying methods of stress tests are complex, and their governance, politics and communication are complex too. Each of these aspects plays a role in the design of stress tests, and they all require us to think deeply and debate thoroughly.
And this event provides the ideal setting for thought and debate. Over the next two days, researchers from around the world will put forward their ideas on many different topics. Among other things, they will discuss how to refine the methods used in stress testing, how to design and calibrate the scenarios, how to assess and model the behaviour of banks, and how to include new risks such as those stemming from climate change.
These are, if you like, the nuts and bolts of stress testing. But today, I will leave it to the researchers to get on with the job of detailed research – and will instead focus on the big picture: how do I as a supervisor see the future of stress tests in Europe?
I gave a speech on the future of stress testing back in September, so I will pick up some of the ideas I discussed then and offer a few more details here and there. My remarks are intended as a contribution to the debate on the future of the European stress test exercise that the European Banking Authority (EBA) is planning to launch. I am well aware of how important the stress test is for the EBA, and am very keen to see a robust debate that results in a stable and effective set-up.
Stress testing European banks: purposes and criteria
The main purpose of stress tests has not changed much in recent years. They aim to assess how banks would fare if the economy took a turn for the worse, estimate how much of their capital would be depleted and check whether they would be able to survive. These are the most general questions that stress tests can help to answer.
But over time, stress tests have served other goals as well. During the financial crisis, they became a key tool in determining how much capital was needed to fill the gaps in banks’ balance sheets. They also improved transparency, by making available a massive set of increasingly comparable data. This reduced uncertainty and helped to calm the markets. In the run-up to the start of the banking union, stress tests helped to level the playing field for supervisory practices and regulatory definitions across the Single Market.
Stress tests have continued to evolve since then and we now see them serving five wide-ranging goals. First, stress tests help supervisors to determine how much capital banks should hold. Second, they assist banks in their efforts to improve their risk management capabilities. Third, they support other supervisory activities, such as on-site inspections. Fourth, they help us to assess the risk profiles and vulnerabilities of banks in a quantitative manner. And fifth, they provide transparency on the risk profile of banks, thus helping to foster market discipline and raise market confidence.
That is an ambitious set of goals for one tool to achieve. Are we trying to do too much with one tool? Well, we have seen first-hand how trying to cover so many goals in one exercise has made the European stress test fairly complex and resource-intensive. And this, in turn, has led to a certain amount of discontent among supervisors and banks alike.
So, the time has come to think about redesigning the stress test in a way that best serves our needs.
Redesigning the stress test – turning one into three
And one of the main questions is: how to reconcile all the competing goals? Well, if you cannot use the same tool for everything, the pragmatic solution would be to use more than one tool. And this is what we propose to do. If one exercise cannot serve all goals, then split it up. Overall, we suggest splitting the stress test into three elements: a supervisory view, a bank view and a macro view.
Not only would this help us to better achieve all the goals I mentioned, it would also help to make the stress test generally more relevant for banks, supervisors and markets. It would ensure that the results can still be compared across banks and that the exercise does not consume too many resources. And it would allow us to maintain adequate transparency.
So what is behind the three views we suggest?
Let’s begin with the supervisory view. The supervisory view would, by and large, be a more efficient version of the current stress test.
That is, it would still be based on a constrained bottom-up approach. Banks would identify their individual starting points and project how the stress scenario would affect their capital positions. These projections would come from banks’ internal models, which would be subject to a set of constraints in order to ensure greater comparability of the results across banks, while also preserving a level playing field when the results are used for supervisory purposes.
Supervisors would then challenge the banks’ results from the supervisory perspective. To this end, they would rely on top-down models, bottom-up benchmarking and the expert knowledge of the teams who supervise the relevant bank.
The results would then serve as the starting point for determining our Pillar 2 guidance. For this purpose, one option might be to sort banks by capital depletion and assign them to buckets. The Pillar 2 add-ons would then be determined on the basis of the bucket to which a bank belongs – similar to the approach for handling the buffer for global systemically important banks. This would acknowledge that the stress test is not such a sophisticated tool that it can determine to the precise basis point how much of a capital buffer a bank needs to withstand a severe shock. As banks would have fewer incentives to fight for an adjustment that could bring down the capital add-ons by a few points, this could also alleviate the process of quality assurance.
And this brings me to my next point. Most of what I just talked about is very similar to the current stress test. So what has changed? Well, the main difference is that because of a shift in focus when publishing the results, the supervisory view requires fewer resources than the current approach. The focus for publication would be on the capital depletion per bank. Compared with the current approach, this would require far less granular quality assurance by supervisors; although, we would, of course, still focus on the specific weaknesses of banks as identified in the stress test. A much smaller amount of data would be published under the supervisory view, while a high level of transparency would be maintained through the less constrained exercise conducted by the banks themselves, which I will come to in a moment. As a result, the quality assurance process would be lighter, as there would no longer be a need for the supervisors to agree with the banks on extremely granular results. Less back and forth would help to bring costs down.
So, the supervisory view would eat up fewer resources than the current stress test – for supervisors and banks alike. At the same time, it would ensure that results can still be compared between banks, thanks to the common constraints placed on all banks’ internal models.
In a way, these constraints act as a safeguard to ensure the credibility of the results and their consistency across banks. This consistency is needed in particular when the results are used to determine capital buffers. The constraints also discourage banks from engaging in a beauty contest where they try to look good instead of real. And banks do try this. In the first rounds of the European stress tests banks did indeed often try to compensate for losses in the adverse scenario, either by being too optimistic when they estimated their income in the adverse scenario or by being very positive about what management could achieve in turbulent market conditions.
And even worse, we have seen several instances of banks colluding to game the stress test, helped by external advisers. Data are collected from banks before they submit them to supervisors, and then each bank is informed of its position vis-à-vis its peers. This helps the banks to align before and during the stress test in order to collectively adjust the results and minimise the impact. We are very determined to avoid a repeat of these practices in future exercises.
But let’s get back to the constraints. While they play an important role in ensuring comparability, they also reduce the realism and thus the relevance of stress tests for banks. To some degree, constraints disguise the true impact of the stress scenario on a bank’s balance sheet. In some cases, constraints might even be seen as the main driver of results rather than as a backstop for credibility. And this could render the stress test much less relevant from a bank’s point of view.
So, if we attempt to make stress tests more realistic and more relevant, the constraints might be one of the levers to pull. Within the supervisory view, we might selectively relax some of the constraints. Among other things, we might rethink whether it is still warranted to assume static balance sheets. Relaxing constraints would enable us to better account for bank-specific factors and for management actions. Results would become more realistic.
But splitting the stress test into three would open up yet another path to greater realism. And this brings us to the second element: the bank view. The bank view would not only help to make the results more realistic and thus more relevant; it would also help to maintain transparency, which would be more limited in the supervisory view.
The bank view would be the new element of the exercise. It would be run in parallel to the supervisory view and would give banks more freedom. They would have to apply the EBA methodology, of course, but would be allowed to relax some of the constraints. Banks could thus better account for their individual risk profile and vulnerabilities. Overall, this would allow them to predict more accurately how the stress scenario would affect their balance sheet.
But to be clear: banks would not be obliged to relax constraints. They could also just revert to the results from the supervisory view. This would allow them to find the right balance between realism and relevance on the one hand and resource intensity on the other.
Banks would then have to publish the results of the bank view at a similar level of granularity to the current stress test. This would ensure continued transparency. Naturally, banks would have to explain which constraints they had relaxed and how. This would ensure that results can still be interpreted in a meaningful way.
Overall, the bank view would provide transparency and it would ensure that results remain realistic and relevant: relevant for banks, as they would be more informative for the bank’s own risk management than the constrained supervisory view, and, at the same time, relevant for markets. This, however, hinges on two conditions. First, the models used by banks have to be sound; second, banks must not adjust results for strategic reasons.
In this sense, the supervisory view would act as a benchmark. As both the bank and the supervisory view would be published at the same time, markets would be able to compare results. They could then gauge whether the bank view was realistic and form their own opinion. Banks would probably be asked to explain the reasons for the differences between the bank view and the supervisory view. So market discipline would be strengthened.
Despite all the benefits I have just mentioned, I am aware, of course, that publishing two sets of stress test results also entails certain challenges. But these challenges are offset by three mitigating factors. First, the target audience should be aware that the bank view and the supervisory view rest on different methodologies. So it should not come as a surprise when results differ. Second, the two views are disclosed at different levels of granularity and so can only be compared at a very high level – the capital depletion. And third, each bank would be provided with the results from the supervisory view before they publish the bank view. This might help them to assess how robust their own results are and to consider the rationale for potential deviations between the two views.
The redesigned stress test would also contain a third element: the macro view. This view would make it possible to simulate further events which are relevant at the macro level.
Importantly, however, the macro view should not provide a third set of stress test results but should rather supplement the supervisory view with a top-down sensitivity analysis. It would help in discerning how robust the results from the supervisory view are at the aggregate level. In other words, it would help illustrate how the results might change if the central scenario were modified or different methodological assumptions were used. This would support the high-level narrative of the supervisory view. It is conceivable that in the not too distant future we will have longer time series of consistent data and more reliable ECB models to estimate very granular risk parameters. This could enable us to use top-down stress tests for supervisory purposes too, as is currently done in the United States. At that stage, the supervisory view and the macro view could be merged into one. But we are not there yet.
Ladies and gentlemen,
The last time I talked about redesigning the stress test, I emphasised that I was just outlining a few ideas. This remains the case, as the debate on the future of stress testing is just starting. This means that the future of stress testing is not yet upon us – in any case, it will not arrive before 2022. In other words, the stress test scheduled for next year will still follow the current approach.
One thing is certain, though. Stress tests have become an indispensable tool for supervisors – I certainly believe in them. But they are not static. First, they are quite a recent addition to the supervisory toolbox, meaning there is still scope to refine them. Second, they need to adapt to the post-crisis world and the changing goals of stress testing. The guiding principles for any future changes to the European stress test, as I see it, should be realism, relevance, cost-efficiency (for both supervisors and the industry) and transparency. If we succeed, the stress test will continue to be an exercise that helps to make banks safer and sounder. And that is the overarching goal.
Thank you for your attention!