On August 30, the Detroit Free Press published an article reporting a sharp decline in median household income from 1999 to 2006.[1] According to the article and its accompanying map, median household income declined 6 percent over the time period, with the median household in some states losing over ten percent of its real income. Political pundits seized on these numbers as proof that President Bush's economic policies have been disastrous for Americans.[2] But the Free Press reporters made a major mistake in tabulating their results. Specifically, they compared different datasets that are simply not comparable, according to the data's publisher, the Census Bureau. Had the Free Press stuck with a single dataset and compared apples to apples, it would have found that median household income is much healthier than it reported.

Mistake #1: Comparing Two Different Datasets

The reporters used data from the 2000 Census and the 2005 American Community Survey (ACS), both published by the Census Bureau. The ACS is a new Census tool that presents income and demographic information for states, congressional districts, and cities. The 2000 Decennial Census will be replaced by the ACS in 2010.

However, the two datasets are not comparable. The Census Bureau clearly warns about the dangers of such a comparison: "[E]stimates from any one survey will almost never exactly match the estimates from any other (unless explicitly controlled), because of differences such as in questionnaires, data collection methodology, reference period, and edit procedures."[3]In fact, the Census Bureau also states that the ACS existed only in test form from 2000 to 2004. In the August release of the 2005 ACS, the Census Bureau wrote, "Since 2005 was the first year that the ACS was fully implemented, this report will not make comparisons with previous years."[4] The ACS is just not old enough to compare results from different years.

The differences in the two datasets are readily apparent in this table (Microsoft Excel link), which compares median incomes from the 2000 Census (which measured 1999 income) and the 2000 ACS and was prepared by Megan McArdle, a reporter for the Economist newspaper. As McArdle explains, "[U]sing this methodology, most states experienced a drop in median income in 2000," a year in which the economy was not so weak.[5] "All [the Free Press] graph really tells us is that the new ACS produces lower estimates of median income than the Census long form," concludes McArdle.

Mistake #2: Using the Wrong Dataset

The Detroit Free Press used the ACS for its story on median income but should have used the Current Population Survey's Annual Social and Economic Supplement (CPS ASEC) instead. The CPS ASEC is the official survey of income that is used by the Census Bureau. Census considers the CPS ASEC to be the best measure of income. According to the Bureau, "The CPS ASEC provides the most timely and most accurate cross-section data for the nation on income and poverty."[6]

The CPS ASEC is a more detailed survey, asking respondents 50 different income-specific questions, compared to the ACS's eight questions. It is not surprising that the CPS ASEC shows far different results from the ACS.

The table below shows the change in median household income in the CPS ASEC during the same time period as the Detroit Free Press analysis. Some states that the Free Press reported showing large decreases in income, such as Utah, actually experienced income gains, while the overall decline in nationwide income dropped by half.

Mistake #3: Using 1999 as a Baseline

1999 was the peak of the previous economic cycle. Real median household income declined in 2000 and for the next four years. The economy naturally fluctuates up and down over the course of the business cycle. Comparing the peak of one cycle to a non-peak year in another confuses natural cyclical changes in the economy with a change in the underlying trend of growth. It is a misleading methodology.

Any comparison that uses 1999 as a starting or ending point for analysis will be flawed unless it compares 1999 to the peak of another business cycle. To present an accurate picture of the economy's recent performance, the Free Press should have compared median income in 2005 with inflation-adjusted income in 1995, four years after the 1991 recession. Had it done so, its analysis would have shown that median household incomes rose 9.0 percent between 1995 and 2005.

Conclusion

The Detroit Free Press compared apples to oranges when it reported that median household incomes have dropped dramatically since 1999. It compared Census data with ACS data, despite Census Bureau warnings against doing this. The ACS reports much lower income levels than the Census data, and so it is unsurprising that the Detroit Free Press found that income fell dramatically. More reliable and consistent data from the CPS ASEC show that median incomes fell less than half as much as the Detroit Free Press reported.

The Detroit Free Press also compared apples to oranges by comparing incomes in 1999, the height of the tech bubble and the peak of the previous economic cycle, with incomes in 2005, only four years from the end of the last recession. Had it looked at growth over comparable time periods, it would have found that median household incomes rose.

Rea S. Hederman, Jr., is a Senior Policy Analyst, and James Sherk is a Policy Analyst, in the Center for Data Analysis at The Heritage Foundation. The authors gratefully acknowledge the contributions and research of Stuart Buck, whose efforts helped shape this paper.