Last October, Congress passed the 1999 Omnibus Appropriations Act, which, among other things, allocates $18 billion to the International Monetary Fund (IMF). The passage of this bill followed an intense debate on the effectiveness of the IMF, as well as its capacity for reform to improve its performance. To measure the IMF's effectiveness objectively, Heritage analysts looked at the IMF's economic forecasts from 1971 to 1998 for various industrial countries and developing regions. These forecasts are published biannually in the IMF's World Economic Outlook (WEO). 2 The following report presents the results of Heritage's analysis.
Assessing the IMF's Forecasting Accuracy
Founded in 1944 to ensure the stability of the international financial system, the IMF has evolved into an organization that primarily bails out economies in distress. For example, it orchestrated bailouts in excess of $22 billion to Russia in 1998, $17 billion to Thailand in 1997, and $50 billion to Mexico in 1995. Its current resources are approximately $300 billion. 3 Among the IMF's 182 members, the United States is the largest contributor with an assessed "quota" of nearly $51 billion. 4 In recent years, the organization has been criticized for continuing to bail out countries that fail to implement reforms attached to previous loans, for distorting investment decisions in failing economies by creating various incentives to borrow too much or invest too little, and for other "moral hazard" problems.
The IMF regularly forecasts major macroeconomic developments in various developed countries and developing regions to monitor the world economy and gauge the effectiveness of its funding programs. The WEO forecasts provide the IMF with an integral frame of reference for its future policy decisions and a base against which to judge how its policies are being implemented. They also play a key role in determining the IMF's funding requests to Congress. The accuracy of these forecasts, then, is important to policymakers in considering appropriations for the organization.
Perhaps the most serious concern surrounding the WEO forecasts involves the potential for bias. Because IMF loans are tied to policy recommendations, its forecasts for each country requesting a loan could mirror the expected outcome of the IMF's policies and this bias would result in forecasts that are too optimistic. A less than positive forecast--one showing no improvement or an even worse economic situation--would indicate that the IMF's programs were ineffective. The IMF's forecasting track record suggests that bias does exist.
The Countries Studied. Heritage analysts studied the WEO forecasts for industrial countries and for developing countries by region. It should be noted that the IMF does not publish its forecasts for individual developing countries; instead, forecasts are given only in the form of aggregated developing regions.
Heritage analysts assessed the IMF's forecasts, as published in the World Economic Outlook, for Canada, France, Germany, Italy, Japan, the United Kingdom, and the United States, as well as the combined regional grouping of G-7 countries, or Major Industrial Countries (MICs), for the years 1971 to 1998. The analysts also assessed the forecasts for regional groupings of developing countries, specifically Africa, Asia, Europe, the Middle East, and the Western Hemisphere, for the period 1977 to 1998.5
The Measurements Studied. Heritage analysts limited their assessments to three key macroeconomic indicators:
-
Output growth - measured by the change in real gross domestic product (GDP);
-
Inflation - measured by the change in the GDP deflator 6 for the industrial countries, and by consumer prices for developing regions; and
- Balance of payments (BOP) - measured by the balance of payments on the current account.
The Findings. The Heritage analysts found that the bias in the WEO economic forecasts varied. In general, forecasts for industrial countries outperformed those for developing countries. Specifically:
1. For Industrial Countries
-
The IMF made unbiased and efficient forecasts for developed countries in terms of real GDP growth, inflation, and balance of payments on the current account. 7
-
Because the industrial countries are more economically and politically stable, they are easier to forecast than developing countries.
- The IMF does a better job of forecasting the industrial countries' output growth and inflation than it does of forecasting their balance of payments on current accounts. But relative to a "random walk forecast" (which assumes that the growth rate in year t equals the growth rate in year t - 1), the accuracy of the IMF forecasts diminishes over time for both real GDP growth and inflation. 8
2. For Developing Regions
-
As IMF funding increases, so does the forecast error (the actual outcome minus the forecast). The significant relationship between IMF funding and the forecast error implies that bias in the forecast depends on whether the country receives IMF funding. For example, for every additional billion in Special Drawing Rights (SDR) the IMF gave to the Western Hemisphere, the forecast error increased by 0.17 percentage points. A similar correlation occurred for inflation in the pooled regions and for the balance of payments in Africa and Asia.
-
"Turning point" errors, in the form of over- and underestimation, are prevalent in the IMF forecasts for developing regions. The forecasts overestimated real GDP growth for the pooled regions by an average of 0.57 percentage points from 1977 to 1998. On the other hand, the forecasts underestimated consumer price inflation by an average of 18.2 percentage points each year.
- For Africa, a random walk forecast of real GDP growth and consumer price inflation predicted outcomes more accurately than did the WEO forecasting model.
ANALYTICAL TESTS APPLIED 9
Previous studies have used statistical and analytical tests to determine the accuracy of IMF forecasts in the WEO. 10 This report builds on those statistical techniques and applications by reviewing a longer period of time and by using more recent data. The analysts tested the accuracy of the WEO forecasts for turning point errors in the form of systematic over- or underestimation, directional accuracy, bias and efficiency, and performance over time relative to a naïve model.
-
Turning Point Errors: Do WEO forecasts systematically over- or underestimate economic outlook? To determine systematic turning point errors in the WEO forecasts, the analysts examined whether the initial forecast and a subsequent revision 11 consistently fell below or above the actual figure. If the initial forecast fell above/below the actual figure and the subsequent forecast revision also fell above/below it, then the WEO systematically overestimated/underestimated the economic outlook for that country.
-
Directional Accuracy: Do WEO forecasts correctly anticipate change? The second test applied to the data examined was whether the WEO forecast correctly anticipated at what point a data series would change direction. For example, suppose inflation in the United States increased one year and declined the following year. A forecast that identified when inflation changed direction would be considered a good forecast because it would have predicted the change in the data series accurately.
- Bias and Efficiency: Are WEO forecasts unbiased and efficient? Heritage analysts examined whether the IMF produced unbiased forecasts. On average, an unbiased forecast would fall very close to the actual outcome. However, since perfect forecasts are not possible, it is necessary to measure how much the forecast deviates from the actual outcome. For example, suppose a forecast overestimated output growth for the U.S. economy by an average of 0.5 percentage points. Is this deviation from the actual outcome too great, or does it fall within a reasonable range? 12 If the deviation falls within a reasonable range, then the forecast is considered unbiased.
Heritage analysts next considered whether IMF forecasts were efficient. They considered a forecast inefficient if 1) the forecast error is related to the forecast itself or 2) the forecast error is related to the forecast error of the previous year. In either case, an appropriate adjustment would improve the forecast. 13 For example, suppose the forecast increases, on average, by 0.1 percentage point when the forecast error increases by 0.1 percentage point. The accuracy of the forecast could be improved by subtracting 0.1 percentage point from the forecast at the time it is made.
- Performance: Do WEO forecasts improve over time? Heritage analysts compared the WEO forecasts with those generated by a naïve model using a simple random walk forecast--which assumes that the growth rate for the current year equals the growth rate from the previous year. Specifically, the analysts compared the error term from the naïve forecast with the error term from the WEO forecasts to determine whether WEO forecasts improved over time.
SYSTEMATIC OVER/UNDERESTIMATION
A closer examination of the WEO forecasts shows the prevalence of systematic turning point errors relative to the actual value. 14 These errors take the form of consistent under- and overestimation, which are pervasive in WEO projections for output growth, inflation, and balance of payments on the current account in both industrial countries and developing regions.
Turning point errors imply that the IMF forecasts fail to capture and include crucial economic events and shocks. This failure would weaken the IMF's effectiveness because early diagnosis of its member countries' vulnerabilities to potential crises is critical to fulfilling the IMF's mandate of ensuring the international financial system's stability. Tables 1a and 1b report the results of this analysis.
As various IMF reports 15 note, IMF projections hinge on an assumption that developing countries with IMF-supported adjustment programs are pursuing IMF-mandated policies to achieve macroeconomic stability and reduce structural distortions. However, given the various response mechanisms and the immensity of the variables at play, the efficacy of the complex and interlocking policies adopted by each government is difficult to determine. Factors such as the availability of data--given the weak data monitoring capabilities of a majority of the developing countries--could also adversely affect the forecasting process, even for the IMF.
Using a simple testing method 16 and discounting the existence of a host of constraining variables present, this analysis found that the IMF was unable to anticipate the following important events adequately:
-
Hyperinflation in the late 1980s. The end of the 1980s witnessed an upsurge of inflationary pressures in some developing countries, such as price increases at an annual rate of at least 200 percent in Argentina, Brazil, and Mexico. The IMF failed to anticipate and then severely underestimated this period of accelerating inflation. For example, WEO forecasts underestimated inflation for the Western Hemisphere region from 1988 through 1990 by an average of 311 percent, peaking in 1990 with a huge error of 455.8 percent. In Europe, the WEO underestimated average consumer prices by as much as 119.5 percent in 1989. The IMF, while acknowledging that the WEO forecasts tended to underestimate inflation, attributes this to policy slippage and delayed implementation of stabilization programs. 17
- Industrial growth slowdown in 1995. Growth among industrial countries slowed in 1995. Not only did the worldwide rise of long-term interest rates in 1994 depress industrial country activity, but the effects of the Mexican peso crisis, the large appreciation of the yen that weakened the incipient Japanese recovery, and the appreciation of some European currencies against the dollar (e.g., the deutsche mark) further exacerbated the economic downturn.
As Table 1a shows, IMF forecasters apparently overlooked the depressing effect of the above factors and were unable to anticipate the downturn. As a result, the WEO forecasts overestimated output growth for industrial countries during 1995, albeit at comparatively small levels (though quite large in percentage terms). In fact, the forecasts overestimated growth for Canada and the United States by 2.1 and 1.2 percentage points, respectively. The U.S. slowdown resulted from a sharp inventory correction and a decline in real net exports that reflected the effects of the recession in Mexico. Canada, on the other hand, was weakened sharply by high interest rates, slow employment growth, falling real disposable incomes, high levels of consumer debt, and cutbacks in government consumption.
;)
- Japan's slowdown and the Asian financial crisis. In the 1980s, Japan experienced a massive transformation due to its economy's rapid growth. Domestic demand and business investment increased at a rapid rate. WEO forecasts, however, failed to identify this trend and consistently underestimated Japan's output growth from 1987 to 1990. The forecasts also underestimated the growing economic strength and rapid industrialization of the so-called Asian tigers, particularly in the early 1990s.
From 1992 to 1998, the IMF's forecasts for Japan's output growth were characterized by systematic overestimation. It can be surmised that the IMF expected that the fiscal stimulus packages and a host of other policy changes adopted by the Japanese government would reinvigorate the economy. However, the IMF did not fully anticipate that the fiscal policies Japan adopted would be more contractionary than expected and that private demand would weaken. This weakness in demand compounded weaknesses in the financial sector and bad loan difficulties, delaying implementation of structural reforms that would reinvigorate the economy. WEO overestimation errors widened and then peaked in 1998. While zero growth was predicted for 1998, given the continued weakness in domestic demand, this prediction proved to be more optimistic than the actual figure since Japan slumped into a negative growth rate in 1998.
- Output growth for developing countries. Table 1b shows a consistent IMF record of overestimating output growth for developing countries. This is particularly true for Africa, where the WEO forecasts overestimated output growth each year. As the IMF acknowledges in its various reports, its projections for developing countries are at risk if assumed policies are not implemented.
;)
DIRECTIONAL ACCURACY
To further determine the efficacy of IMF forecasts, Heritage analysts tested the relationship between the sign of the forecasts (i.e., a positive or negative change over the previous actual value) and the current actual values. An important gauge of the forecast's ability to determine turning points is its success in maintaining directional accuracy.
Heritage analysts tested the directional accuracy of the WEO forecasts through a nonparametric method of assessment using the hypothesis that the forecasts and the actual values are independent 18 The directional quality of the WEO forecasts is judged acceptable if 1) the forecasts post an accuracy rate of 70 percent or higher and 2) a significant association between the signs of forecasts and realizations is found. 19 Table 2 shows the results of this test.
;)
As expected, the results show that the IMF's forecasting accuracy for industrial countries is much better than it is for developing countries. Results for both output growth and inflation for industrial countries show that WEO forecasts successfully predict direction of change. Except for the inflation forecasts for Canada, the percentage of correct forecasts on output growth and inflation for the industrial countries exceeds the benchmark level of 70 percent--posting as high as 96 percent accuracy for some countries. The absence (or minimal presence, if any) of IMF funding support for developed regions leads to more objective forecasts unencumbered by any expectation of success intrinsic to the IMF programs.
However, estimations of the balance of payments for industrial countries appear to be of poorer quality compared with output growth and inflation. Forecasts made for the United States, Canada, Germany, and other MICs posted poor results. The WEO forecasts failed to predict changes in the data series for inflation for all these countries or regions at least 30 percent of the time.
In general, the directional accuracy of WEO forecasts for developing regions appears to be weak. The Heritage analysis shows that forecasts for developing regions hardly pass muster for all variables. In fact, they fail for almost all of the regional groupings. Although other factors may be present (such as lack of reliable and quality data), the results are highly indicative of the decreased objectivity of WEO forecasts for developing countries.
The WEO forecasts for developing regions apparently depend on the success of the implementation of IMF assistance. Should the IMF policy intervention fail, or the recipient developing country fail to reach IMF-set targets, there is a high probability that IMF forecasts will be off-target. The results appear to support the hypothesis that the IMF's forecasting for developing regions suffers from an inherent bias toward positive results of its programs.
;)
;)
;)
TESTS FOR BIAS AND EFFICIENCY
Charts 1 through 3 give an overall picture of the performance of WEO forecasts for the MICs and developing regions. 20 The charts on real GDP growth, inflation, and balance of payments show that the WEO forecasts approximate the actual outcome for the MICs. 21 However, for developing regions, 22 the WEO forecasts overestimated real GDP growth and underestimated consumer prices. No conclusion could be drawn regarding the balance of payments on the current account.
Heritage analysts performed a number of statistical tests to examine whether the differences between the actual outcome and the WEO forecasts were statistically significant. Tables 3 and 4 report the summary results for the industrial countries and developing regions respectively. These tables show the average actual value, the average forecast error, and two tests for efficiency. For example, Table 3 shows that U.S. real GDP grew at an annual average rate of 2.6 percent between 1971 and 1998, and the average forecast error was -0.09 percent.
To determine whether the -0.09 percent deviation from the actual outcome signifies bias or the absence of bias, Heritage analysts regressed a constant term on the error term, which yielded the average forecast error. 23 If the average forecast error is negative and significant, a positive bias exists. 24 In Tables 3 and 4, the average forecast errors with an asterisk next to them indicate a high probability (greater than 90 percent) of being significantly different from zero, or biased.
Table 3 shows that the IMF made unbiased WEO forecasts for real GDP growth, inflation, and balance of payments on the current account for the MICs. The only exceptions were inflation for Italy and real GDP growth for the pooled countries. 25 For Italy, Table 3 shows that the WEO forecasts underestimated inflation by an average of 0.55 percentage points. Similarly, the WEO forecasts overestimated real GDP growth for the pooled industrial countries by an average of 0.18 percentage points.
;)
Table 3 also reports on the efficiency of the WEO forecasts. An efficient forecast incorporates all the available information at the time the forecast is made. Two statistical tests are performed to test the efficiency of the WEO forecasts. First, the forecast error is regressed on the forecast itself; this is called the "current error coefficient." If a significant relationship exists between the forecast and the forecast error, the forecast can be improved by making an appropriate adjustment. For example, if the relationship between them equals 0.1, subtracting 0.1 from the forecast will improve its accuracy. Second, the error term is regressed from the previous year on the error term in the current year, called the "lagged error coefficient." Again, no significant relationship should exist between these two variables.
Table 3 shows that the WEO forecasts were efficient for real GDP growth in industrial countries, as indicated by the coefficient on both the current error and the lagged error being insignificant. However, forecasts for inflation in Canada, the United Kingdom, MICs, and pooled countries are inefficient, as indicated by the coefficient on the current error or the lagged error being significant. Forecasts for the balance of payments on the current account are less efficient than the output and inflation forecasts.
Table 4 reports the findings for bias and efficiency in the forecasts for developing regions. The results show that the IMF made unbiased forecasts for real GDP growth in these regions, with the exception of Africa. For the period 1977 to 1998, the forecasts overestimated GDP growth by an average of 0.57 percentage points (significant at the 5 percent level) when data for the developing regions are pooled together. The IMF overestimated real GDP growth for Africa by an average of 1.05 percentage points (significant at the 1.0 percent level) each year.
;)
All of the regions, except the Middle East, exhibit a significant bias for consumer price inflation, which the IMF generally underestimated. The most dramatic example occurred in the Western Hemisphere, where the forecasts underestimated inflation by 67 percentage points over the years 1980 to 1998. Forecasts for balance of payments on the current account are unbiased, except for the Middle East and Western Hemisphere regions.
The IMF made efficient WEO forecasts for real GDP growth for the developing regions, with the exception of Europe and the pooled regions, where both the current error coefficient and the lagged error coefficient are significantly different from zero. Approximately half the forecasts for consumer price inflation and balance of payments on the current account are efficient. Only Africa and Asia pass the efficiency tests for consumer price inflation. Europe and the Middle East pass the efficiency test for balance of payments on the current account.
A comparison of the results in Table 3 and Table 4 shows that the WEO forecasts for industrial countries outperform those for the developing regions. Forecasts for industrial countries are generally unbiased and more efficient than those for developing regions.
More important, the results seem to confirm the hypothesis that the IMF may have a bias in formulating its forecasts for developing regions. As mentioned earlier, there appears to be a strong incentive for the IMF to produce forecasts that support its policy positions, primarily because the IMF gives funding to and makes policy recommendations for these developing regions. By contrast, because the IMF rarely provides any funding to industrial countries, it has little incentive to produce overly optimistic forecasts.
Given the varying incentives it faces in making forecasts for the developing regions and the industrial countries, there is a strong incentive for the IMF to overestimate real GDP growth and underestimate inflation for the developing regions. A comparison of the tests on inflation in Tables 3 and 4 provides evidence that this indeed is occurring. For industrial countries, the WEO forecasts were unbiased for inflation; the signs on the average forecast error indicate that inflation is as likely to be overestimated as underestimated. Conversely, the WEO forecasts were significantly biased for inflation in the developing regions, and the signs on the average forecast error indicate that WEO forecasts underestimated inflation .
PERFORMANCE OVER TIME
Tables 5 and 6 compare how well WEO forecasts perform over time relative to the random walk forecast for industrial countries and developing regions. Heritage analysts considered two test statistics: the root mean square error (RMSE) and the Theil Inequality Statistic. The RMSE is the square root of the average value of the errors squared. The RMSE is preferred to other measures of the error term, like the sum of the average absolute value of forecast errors, for example, because it gives a greater weight to large forecast errors than to small ones. Although a lower RMSE indicates a smaller error, there are no established rules for distinguishing an acceptable RMSE from one that is unacceptable.
The Theil statistic is a more useful measure. In this context, the Theil statistic is the ratio of the RMSE of the WEO forecast to the RMSE of the random walk forecast. 26 A Theil statistic greater than 1.0 indicates that the random walk forecast predicts the actual outcome better than does the WEO forecast. Because the Theil statistic is a ratio, it will be greater than only 1.0 when the numerator (the RMSE of the WEO forecast) is greater than the denominator (the RMSE of the random walk forecast). 27
Table 5 compares the RMSE and Theil statistic for industrial countries for the periods 1971-1984, 1985-1998, and 1971-1998 28 . Table 5 shows that the WEO forecasts outperform the random walk forecasts over the entire sample period. Most of the Theil statistics are below 1.0, except for inflation in Canada from 1985 to 1998 and balance of payments on the current account for Canada from 1971 to 1984. These results indicate that the WEO forecasts outperform the naïve forecast.
;)
Heritage analysts evaluated WEO performance over time by examining changes in the forecast error. If the forecast error diminished over time, it would appear that the WEO forecast had improved. This can be misleading because the economic environment may have changed, making it easier to forecast. To account for this possibility, the analysts compared the RMSE and the Theil statistics in different time periods. If the RMSE of the WEO forecast decreased in the same proportion as the RMSE of the naïve model (that is, the Theil statistic did not improve), then it can be inferred that the quality of WEO forecasts did not improve. 29
Comparing results from 1971-1984 and from 1985-1998 demonstrates that the RMSE improves for every industrial country for real GDP growth and inflation but worsens for balance of payments on the current account. The Theil statistic increased in 18 out of 24 cases, indicating that the forecasts made in the WEO had not improved relative to the random walk forecast. Although the IMF forecast error has diminished over time, it is likely due to the greater stability of the time series.
Table 6 suggests that forecasts for the developing regions have improved marginally over time. 30 Comparing the results across the two sample periods reveals that the Theil statistic decreases in 6 out of the 9 forecasts and that the RMSE increases in 7 out of the 9 forecasts. These results suggest that the RMSE of the WEO forecast has increased, but by a lesser proportion than the RMSE of the naïve forecast, which implies that WEO forecasts have improved marginally over time.
;)
IMF FUNDING AND WEO FORECASTS
Because the IMF provides funding and makes policy recommendations to a country, its WEO forecasts are likely to support its policies. The IMF recognizes that some form of bias may exist.
The staff's projections are generally based on the assumption of broadly unchanged policies. However, in certain cases where significant policy changes are considered likely--for example, in the context of Fund- or Bank-supported adjustment programs--policies are projected to improve in line with program objectives. In view of the slippages that have repeatedly occurred in a number of countries, this assumption could entail considerable downside risk for some projections. 31
The existence of this bias--where forecasts follow the direction of the anticipated effect of IMF policies--narrows the range of forecasting possibilities. Forecasts also become overly optimistic, resulting in larger forecast errors than warranted. Thus, the quality of IMF forecasts ultimately may suffer from this inherent bias.
To examine whether the forecast error increases as IMF funding to a region increases, Heritage analysts regressed IMF funding on the forecast error for the developing regions. Table 7 shows that increases in funding did lead to an increase in the error term in real GDP growth for the Western Hemisphere and the pooled regions. As noted above, for every additional billion in Special Drawing Rights the IMF gave to the Western Hemisphere, the forecast error increased by 0.17 percentage points. A similar correlation occurred in inflation for the pooled regions and in the balance of payments for Africa and Asia.
Table 7 also reports a relation noted as "R2" for each of the regressions. This statistic is used to measure how much the IMF funding explains the variation in the forecast error. For example, IMF funding explains approximately 20 percent of the variation in the forecast error for Asia's balance of payments on the current account, but it explains zero percent of the variation in the forecast error for consumer price inflation. Although the R2s reported in Table 7 are very low, it does appear that IMF funding contributes to the forecast error.
;)
CONCLUSION
Although further tests need to be performed, Heritage analysts found evidence of inherent bias in IMF forecasts. IMF forecasts for developing regions were overly optimistic: The WEO forecasts overestimated output and underestimated inflation. The WEO forecasts for real GDP growth for Africa and inflation for the Western Hemisphere demonstrate this bias most clearly. The IMF overestimated real GDP growth for Africa by an average of 1.05 percentage points each year and underestimated inflation by 67 percentage points each year in the Western Hemisphere. In addition, the analysts found that, as IMF funding increases, so does the error term. This result suggests that the incentive to make an overly optimistic forecast increases as IMF funding to a region increases.
An alternative way to test the hypothesis that the IMF makes biased forecasts would be to compare the forecast errors before and after a country receives IMF funding. If the forecast error in the WEO figures was significantly larger after a country received IMF funding, that will indicate an inherent bias. However, testing this hypothesis will require the use of unpublished IMF forecasts for individual developing countries.
In conjunction with the International Monetary Fund's commitment to becoming more transparent, forecasts for individual developing countries should be made readily available for independent analysis. This accessibility would allow researchers to evaluate more conclusively whether there is an inherent bias in the IMF's forecasting of economic development in developing countries.
William W. Beach is Director, Aaron B. Schavey is a Policy Analyst, and Isabel M. Isidro is a Research Assistant in the Center for Data Analysis at The Heritage Foundation.
APPENDIX:
METHODOLOGY
DATA
Heritage economists evaluated the accuracy of forecasts made by the International Monetary Fund and published in its World Economic Outlook, based on the properties of the forecast error. For these properities and other information on the data used to produce this analysis, please click here for the PDF file.
2. From 1980 to 1983, the WEO was published annually. Since 1984, it has been published biannually in May and October. See also Michael J. Artis, "How Accurate Are the IMF's Short-Term Forecasts/ Another Examination of the World Economic Outlook," IMF Working Paper No. WP/96/89, August 1996, for IMF's forecasts from 1971 to 1979.
3. Exchange rate as of August 4, 1999: US$1.368 to 1 SDR. See International Monetary Fund Web site at http://www.imf.org/external/np/tre/sdr/drates/0701.htm. SDRs (Special Drawing Rights) are units of account used for IMF transactions and operations.
4. See International Monetary Fund at http://www.imf.org.
5. For developing regions, data constraints limit the time series used from 1977 to 1998, except for the Middle East and Europe (1977-1991). These two regions were aggregated beginning in 1992, using data taken from Artis, "How Accurate are the IMF's Short-Term Forecasts/ Another Examination of the World Economic Outlook."
6. The GDP deflator from 1971 to 1998 offers a more comprehensive data set for industrial economies.
10. See Artis, "How Accurate Is the WEO? A Post Mortem on Short Term Forecasting at the IMF," Staff Studies for the World Economic Outlook, International Monetary Fund, Washington, D.C., 1988, and "How Accurate Are the IMF's Short-Term Forecasts? Another Examination of the World Economic Outlook." See also Jose M. Barrionuevo, "How Accurate Are the WEO Projections?" in Staff Studies for the World Economic Outlook, December 1993, and Harjit K. Arora and David J. Smyth, "Forecasting the Developing World: An Accuracy Analysis of the IMF's Forecasts," International Journal of Forecasting, No. 6, 1990, pp. 393-400; Michele Fratianni and John Pattison, "International Institutions and the Market for Information," in Roland Vaubel and Thomas D. Willet, eds., The Political Economy of International Organizations: A Public-Choice Approach (Boulder, Colo.: Westview Press, 1991).
11. Since 1986 the IMF has provided forecasts for a given year and the subsequent year. This is defined as forecast t and t + 1.
12. A forecast is considered to fall within a reasonable range if it can be shown that the forecast error (defined as the actual outcome minus the forecast) is not statistically different from zero.
14. The actual values are defined as the first available economic data released by the IMF. For instance, the first available real economic data for 1998 was released in the May 1999 edition of the WEO.
20. Only regional charts are shown to give an overall summary. Charts for specific countries are available upon request.
21. The MICs include Canada, France, Germany, Italy, Japan, the United Kingdom, and the United States.
22. The developing regions include Africa, Asia, Europe, the Middle East, and the Western Hemisphere.
23. The analysts chose to compute the average forecast error in this way because the regression also determines the significance of the number. If a forecast is unbiased, the forecast error will not be significantly different from zero (i.e. on average, the forecast equals the actual outcome ).
24. The forecast error is equal to the actual outcome minus the forecast. A forecast that is larger than the actual outcome indicates a positive bias represented by a negative error term.
25. Pooling data means that the analysts created a data set by combining data from various countries or regions . There are two pooled data sets: 1) for industrial countries with 224 observations and 2) for the five developing regions with 90 observations.
26. See H. Theil, Economic Forecasts and Policy (Amsterdam: North Holland Publishing Co., 1961) and Applied Economic Forecasting (Amsterdam: North Holland Publishing Co., 1966).
27. See Artis, "How Accurate Are the IMF's Short-Term Forecasts? Another Examination of the World Economic Outlook."
28. Due to data limitations, balance of Payments covers the years 1973-1985, 1986-1998, and 1973-1998.
