Just hours before a congressional hearing this week related to the high costs of regulation, the Environmental Protection Agency (EPA) released a report purporting to show that the Clean Air Act (CAA) Amendments of 1990 will avert 230,000 premature deaths and yield economic benefits totaling $2 trillion in the year 2020.[1] The report is astonishing for a variety of reasons—not the least of which is the enormous discrepancy between the Obama Administration’s numbers and those of a similar previous study by the Clinton Administration EPA, which pegged the economic benefits of the act to be $170 billion (or 91 percent less than the Obama EPA’s estimates). This magnitude of difference is explained by the unreliable assumptions underlying the Obama EPA’s wildly inflated claims.

Nevertheless, newspaper headlines across the country—and throughout the blogosphere—trumpeted the new cost–benefit calculation as proving regulation to be unquestionably beneficial.[2] The media’s lack of scrutiny is particularly troublesome because, in this instance, the EPA is evaluating itself. Indeed, for every step beyond the agency’s press release, the questionable methodology and leaps of logic are painfully obvious.

Poor Methodology

The latest report is the third in a periodic series required under the CAA Amendments of 1990. It comes amid escalating concern that regulation in general—and environmental regulation in particular—is imposing an onerous and unsustainable burden on the U.S. economy and eroding citizens’ fundamental freedoms.[3]

The study was designed to evaluate the benefits of cleaner air to human health and the environment. To do so, the researchers[4] compared two hypothetical scenarios based on conditions in 1990: One scenario “modeled” steep declines in emissions of particulate matter and ozone as a consequence of the regulations imposed by the CAA Amendments of 1990. The difference in air quality between 1990 and 2020 was then used to project improvements in health and the environment. This result was compared to a second scenario in which the scope and stringency of regulatory controls remained fixed at their 1990 levels—i.e., prior to implementation of the 1990 amendments—while allowing for increased emissions from economic and population growth.

Predicting the future effects of regulation can be exceedingly complex given the array of confounding factors at play and thus the multitude of assumptions that must be employed. The benefit estimates in the report range from $250 million to $5.7 trillion—a vast difference that indicates vast uncertainty about the EPA’s claims.

“No Way to Validate”

The researchers acknowledge that “there is no way to validate”[5] their forecast of air quality conditions for the non-CAA scenario, effectively rendering the comparison between regulation and non-regulation meaningless. Moreover, it is simply preposterous to assume that air quality would worsen unabated over the course of 30 years in the absence of a particular statute. History has proven otherwise, of course. Long before the original CAA was enacted in 1963, industrial emissions were declining as a result of technological advances and efficiency improvements.[6] And both factors, as well as others, will continue to drive environmental improvements regardless of regulation.

The research design is only one of myriad flaws underlying the EPA’s claims. In fact, 14 elements of the study that bear directly on the valuation of regulatory benefits are unreliable and constitute “major uncertainties”—i.e., differences in benefit estimates of $100 billion or more, according to the authors of the report.

The three most “significant” of the major uncertainties relate directly to the calculation of lives saved by regulation, which accounts for the largest proportion of economic benefit and thus the basis of the agency’s contention that regulatory benefits dwarf costs. Simply put, the EPA’s claim that the CAA Amendments of 1990 will save 230,000 lives and generate $2 trillion in economic benefits in 2020 is rife with “significant” and “major uncertainties,” according to the authors of the report.

Faulty Assumptions

On the cost side of the equation, the authors acknowledge that the degree of uncertainty associated with “many of the key factors … cannot be reliably quantified.”[7]

The researchers are forthright about their “inability to conclusively state that [particulate matter (PM)] mortality outcome is causal.” In fact, their own analysis of uncertainty found that the benefits calculation for averting PM-related deaths—the largest portion of benefits in the study—could be 84 percent less or 79 percent more, depending upon which of 14 assumptions is used to calculate the health effects of exposure to particulate matter. This uncertainty is compounded by the applied assumption that all individuals are exposed to equal amounts of pollution, and they all respond alike—both of which are factually inaccurate. As noted by the authors, “people living in particular locations may experience much higher or much lower exposures or exposure changes than people in other locations.”

Compared to the Clinton Administration’s calculations, the Obama Administration increased by 50 percent the effect of particulate matter on premature mortality. However, the researchers acknowledge that “the agency has not yet conducted a rigorous quantitative analysis to assess the impact” of this and other methodological differences.

Major Uncertainty

The researchers also acknowledge “major uncertainty” about the timing of change in mortality risk following a change in PM exposure. This is a crucial piece of the benefits formulation, because an immediate impact versus a prolonged effect will determine the value (or benefit) of reducing that exposure. As the summary report states, models that “assume that a substantial proportion of the risk reduction occurs many years after the air quality improvements can lead to significantly lower estimates for the economic value of that improvement. Conversely … models which assume most or all of the risk reduction occurs shortly after the air quality change can result in higher benefit estimates.”[8] According to the researchers, the uncertainty on this point means the benefit estimate could be up to 48 percent less than the EPA claims or 47 percent more.

The largest proportion of economic benefit is based on the value of avoiding premature mortality. Yet the valuation of this benefit ranks among the most significant uncertainties in the study, according to the researchers. Alternative estimates they cite would lower the benefit calculation by up to 22 percent.

Regulatory Victory?

Given these and other uncertainties that plague the report, the findings amount to little more than “guesstimates.” Rather than document that the benefits of regulation roundly trump regulatory costs, the EPA’s report raises serious questions about the agency’s politicization of science. Indeed, it is troubling to witness EPA Administrator Lisa Jackson declaring regulatory victory when the researchers themselves assert the unreliability of their findings. Rather than strengthen its case for yet more regulation, the EPA’s misstatements warrant increased scrutiny of agency actions and increased skepticism about the supposed benefits of the regulatory status quo.

Diane Katz is Research Fellow in Regulatory Policy in the Thomas A. Roe Institute for Economic Policy Studies at The Heritage Foundation.