Complutense University Library

A statistical test for forecast evaluation under a discrete loss function


Eransus, Francisco J. and Novales Cinca, Alfonso (2011) A statistical test for forecast evaluation under a discrete loss function. [ Documentos de trabajo del Instituto Complutense de Análisis Económico (ICAE); nº 07, 2011, ] (Unpublished)

Creative Commons Attribution Non-commercial.


Official URL:


We propose a new approach to evaluating the usefulness of a set of forecasts, based on the use of a discrete loss function defined on the space of data and forecasts. Existing procedures for such an evaluation either do not allow for formal testing, or use tests statistics based just on the frequency distribution of (data , forecasts)-pairs. They can easily lead to misleading conclusions in some reasonable situations, because of the way they formalize the underlying null hypothesis that "the set of forecasts is not useful". Even though the ambiguity of the underlying null hypothesis precludes us from per-forming a standard analysis of the size and power of the tests, we get results suggesting that the proposed DISC test performs better than its competitors.

Item Type:Working Paper or Technical Report
Uncontrolled Keywords:Forecasting Evaluation, Loss Function
Subjects:Social sciences > Economics > Econometrics
Series Name:Documentos de trabajo del Instituto Complutense de Análisis Económico (ICAE)
ID Code:12521

Ash, J.C.K., Smyth, D.J and Heravi, S.M. (1998). Are OECD Forecasts Rational and Useful?: a Directional Analysis, International Journal of Forecasting 14, 381-391.

Greer, M. (2003). Directional Accuracy Tests of Long-Term Interest Rate Forecasts, International Journal of Forecasting 19, 291-298.

Henriksson, R.D., Merton and R.C. (1981). On Market Timing and Investment Performance. II: statistical procedures for evaluating forecasting skills, Journal of Business 54, 513-533.

Joutz, F. and Stekler, H.O. (2000). An Evaluation of the Predictions of the Federal Reserve, International Journal of Forecasting 16, 17-38.

Kolb, R.A. and Stekler, H.O. (1996). How well do Analysts Forecast Interest Rates?, Journal of Forecasting 15, 385-394.

Leitch, G. and Tanner, J.E. (1995). Professional Economic Forecasts: Are they Worth their Costs?, Journal of Forecasting 14, 143-157.

Merton, R.C. (1981). On Market Timing and Investment Performance. I: an Equilibrium Theory of Value for Market Forecasts, Journal of Business 54, 363-406.

Mills, T.C. and Pepper, G.T. (1999). Assessing the Forecasters: an Analysis of the Forecasting Records of the Treasury, the London Business School and the National Institute, International Journal of Forecasting 15, 247-257.

Oller, L. and Bharat, B. (2000). The Accuracy of European Growth and In‡ation Fore-casts, International Journal of Forecasting 16, 293-315.

Pesaran, M.H. and Timmermann, A. (1992). A Simple Nonparametric Test of Predictive Performance, Journal of Business and Economic Statistics 10, 461-465.

Pons, J. (2000). The Accuracy of IMF and OECD Forecasts for G7 Countries, Journal of Forecasting 19, 53-63.

Schnader, M.H. and Stekler, H.O. (1990). Evaluating Predictions of Change, The Journal of Business 63, 1, 99-107.

Stekler, H.O. (1994). Are Economic Forecasts Valuable?, Journal of Forecasting 13, 495-505.

Deposited On:04 Apr 2011 08:58
Last Modified:12 Mar 2014 11:02

Repository Staff Only: item control page