A Response to Queen Elizabeth’s Question on the Global Financial Crisis

Dave Turner, Head of the Macroeconomic Analysis Division, OECD Economics Department

“Why did no one see it coming?” was the disarmingly blunt question asked by Queen Elizabeth in the aftermath of the global financial crisis. A number of economists (some with greater reliance on hindsight than others) claimed that there had been “worrying developments” in finance and the global economy for a long time prior to the crisis. Nevertheless, in the eyes of many, the economics profession was damned by the failure of mainstream forecasters, including the OECD and IMF, to predict the crisis. This failure should not, however, have been surprising; forecasters always had a poor track record in predicting economic downturns, particularly because getting the timing right is so difficult [1]. However, they could do a much better job in conveying what those “worrying developments” mean in terms of risks surrounding their forecasts.

Recent OECD research has found that rapid growth in housing-market and credit-related variables can be useful as early warning indicators of severe downturns [2]. These indicators are also correlated with large forecast errors of GDP growth related to (failures to predict) downturns and so can be used to assess the uncertainty surrounding a forecast, distinguishing between a “safe” regime, where forecast errors are expected to be symmetrical, and a “downturn-risk” regime where errors are more likely skewed to the downside [3]. These distributions can then be used to design a fan chart around the central forecast to provide a visual representation of the risks and uncertainties.

What would such fan charts have looked like just prior to the crisis? As an illustration, a series of fan charts are computed around OECD forecasts of GDP growth for the United Kingdom, one of the major economies most severely affected by the crisis. Each fan chart is represented by a series of successively lighter shaded prediction intervals, so that the probability that the outturn lies within successive intervals is assessed at 50%, 70% and 90%.  A first fan chart is constructed as a ‘straw man’, being based on historical forecast errors assuming symmetry in the underlying distribution of errors and ignoring the early warning indicators. On this basis, the outturn for 2009 GDP growth at almost -5% is well outside even a 90% prediction interval on the fan chart (panel A). An alternative asymmetric fan chart (panel B), which takes into account a domestic early warning alarm for rapid credit growth, implies the outturn is closer to, but still outside, the lower 90% prediction limit of about -4%. Thus, perhaps unsurprisingly, to encompass the extreme negative outturn, it is essential to take account of the international dimension of the crisis. Indeed, in the first half of 2008 early warning alarms were flashing for all G7 countries except Japan and Germany. A third fan chart, whereby the skew is calculated on the basis of both domestic early warning alarms and alarms in other major OECD countries, encapsulates the outturn, which falls within the 50-70% prediction interval (panel C). Moreover, similar fan charts computed for other G7 countries confirm that taking into account the early warning alarms, ensures that fan charts are much better at encapsulating the crisis outturn.

Despite ongoing efforts to improve forecasting models [4], it is likely that forecasters will continue to struggle to predict the timing of future downturns. However, by monitoring credit and housing-market developments and other early warning indicators, they could do a better job of both assessing and presenting the risks surrounding their forecasts. Indeed, perhaps the best possible outcome would be if downturn warnings proved to be false alarms because policy-makers heeded the warnings and took appropriate early action.

Further reading

[1] The following studies provide evidence of the poor performance of all forecasters in predicting downturns:
Abreu, I. (2011), “International Organisations’ vs Private Analysts’ Forecasts: An Evaluation”, Banco de Portugal Working Papers, 20/2011, July.

Fildes, R. and Steckler, (2002), “The state of macroeconomic forecasting”,  Journal of Macroeconomics, 24(2), pp. 435-468.

Loungani, P. (2001), “How accurate are private sector forecasts? Cross-country evidence from consensus forecasts of output growth”, International Journal of Forecasting, 17(3), pp. 410-432.

[2] This paper describes recent OECD work to evaluate the usefulness of early warning indicators of downturns in OECD economies:
Hermansen, M. and O. Röhn (2016), “Economic Resilience: The Usefulness of Early Warning Indicators in OECD Countries“, OECD Journal: Economic Studies, No. 1, Vol. 2016, Issue, 1, pp. 9-35.  OECD Publishing, Paris.

[3] The following paper provides further discussion of the rationale for, and details underlying, the design of the fan charts referred to in this post:
Turner, D. (2017), “Designing Fan Charts for GDP Growth Forecasts to Better Reflect Downturn Risks“, OECD Economics Department Working Papers, No. 1428, OECD Publishing, Paris.

[4] This study draws lessons from the financial crisis which have been, or are in the process of being, reflected in OECD forecasts:
Pain, N. and C. Lewis (2014), “Lessons Learned from OECD Forecasts During and after the Financial Crisis“, OECD Journal: Economic Studies, No. 5, Vol. 2104, Issue, 1, pp. 9-39, OECD Publishing, Paris.

This paper describes the current process of how models are combined with judgement in determining OECD forecasts:
Turner, D. (2016), “The Use of Models in Producing OECD Macroeconomic Forecasts“, OECD Economics Department Working Papers, No. 1336, OECD Publishing, Paris.