Models & quantification

Backtesting

Backtesting is the comparison, after the event, of a risk model's predictions with actual outcomes, to establish whether the model estimates risk correctly. A well-known example is the daily comparison of Value at Risk with the actual change in value of a trading portfolio. Backtesting is part of model validation, not the whole of it.

Source: Regulation (EU) No 575/2013 (CRR) Articles 366 and 369 (version in force on 8 July 2024, applicable until 1 January 2027 under Article 520a), Articles 185, 325bf and 325bi; Directive 2013/36/EU (CRD) Article 101(3); Directive 2009/138/EC (Solvency II) Article 124

For banks with an internal market risk model, backtesting is a legal requirement with direct capital consequences. Until 31 December 2026, under Article 366 of Regulation (EU) No 575/2013 (CRR) in the version in force on 8 July 2024, a bank counts its overshootings over the most recent 250 business days: days on which the change in the portfolio's value exceeded the calculated one-day Value at Risk, measured on both hypothetical and actual changes in value. With five or more overshootings, an addend of 0.40 to 1.00 is added to the multiplication factor of at least 3, and the bank notifies each overshooting to the supervisor no later than within five working days.

From 1 January 2027 the new market risk rules apply. Backtesting then takes place per trading desk: a desk meets the requirements if, over the most recent 250 business days, there are no more than 12 overshootings of the Value at Risk at the 99th percentile and no more than 30 at the 97.5th percentile, on both hypothetical and actual changes in value (CRR Article 325bf). Where backtesting shows that a model is no longer sufficiently accurate, the supervisor reviews the conditions for its use or imposes measures to improve it promptly (Article 101(3) of Directive 2013/36/EU, CRD).

Backtesting is also used outside market risk. For credit risk models under the IRB approach, a bank regularly compares realised default rates per rating grade with estimated probabilities of default, analyses deviations and updates that analysis at least annually (CRR Article 185). Insurers with an internal model test the results against experience (Article 124 Solvency II). Backtesting by definition looks back: a model that performs well over a calm period says little about a crisis. That is why, for internal market risk models, the CRR also requires a rigorous stress testing programme alongside backtesting (Article 325bi).

Last updated: · Editor: ONE Risk Advisory

Frequently asked questions

What is the difference between backtesting and model validation?

Backtesting compares predictions with realised outcomes. Model validation is broader: it also assesses whether the model is conceptually sound, whether the assumptions fit and whether the data are correct. Solvency II requires an analysis of stability and sensitivity alongside testing against experience (Article 124), and for internal market risk models the current CRR framework states explicitly that validation is not limited to backtesting (Article 369).

What happens if a bank has too many overshootings?

The capital requirement for market risk rises through a higher multiplication factor. Until the end of 2026, five or more overshootings in 250 business days add an addend to the factor of at least 3 (CRR Article 366). From 2027, a trading desk with too many overshootings no longer meets the backtesting requirements, and the supervisor can review the conditions for using the model or impose measures.

← All terms