Across the nation, firefighters perform vital services that make communities safer. While mainly funded by local governments and, in many communities, staffed by unpaid volunteers, the federal government has traditionally had no direct role in funding local fire departments.
In 2000, the Fiscal Year 2001 National Defense Authorization Act established the Assistance to Firefighters Grant (AFG) Program, also known as the fire grant program, to subsidize the routine activities of local fire departments and emergency medical service (EMS) organizations.[1] The several types of fire grants are administered by the U.S. Fire Administration of the Federal Emergency Management Agency (FEMA), which is part of the Department of Homeland Security (DHS). In addition to subsidizing the firefighting and emergency response needs of fire departments and emergency medical service organizations, the fire grant program offers Fire Prevention and Safety (FP&S) grants and Staffing for Adequate Fire and Emergency Response (SAFER) grants. The FP&S grants fund projects to improve the safety of firefighters and the public from fire and related hazards. These grants are intended to target high-risk populations and alleviate high incidences of fire casualties. The SAFER grants are intended to increase staffing levels by funding the salaries of career firefighters and paying for recruitment activities for volunteer fire departments.
This report evaluates the effectiveness of fire grants. In other words, this report assesses the marginal effect of the federal contribution to local fire departments. This report does not assess the absolute or overall effectiveness of fire departments.[2]
From fiscal year (FY) 2001 to FY 2009, Congress appropriated $5.7 billion for fire grants.[3] The analysis reported here employs grant award data matched to the National Fire Incident Reporting System (NFIRS), an incident-based database of fire-related emergencies reported by fire departments. In addition, the grant data were matched to data from the Census Bureau, Bureau of Economic Analysis, and Bureau of Labor Statistics. Using panel data from 1999 to 2006 for more than 10,000 fire departments, this evaluation uses fixed-effects regressions to estimate the impact of fire grants on four different measures of fire casualties: firefighter deaths, firefighter injuries, civilian deaths, and civilian injuries.
Overall, this impact evaluation finds that fire grants, including grants that subsidize the salaries of firefighters, had no impact on fire casualties:
AFG grants used to purchase firefighting equipment, vehicles, and fitness equipment failed to reduce firefighter deaths, firefighter injuries, civilian deaths, or civilian injuries.
FP&S grants, which funded fire prevention and safety projects, failed to reduce firefighter deaths, firefighter injuries, civilian deaths, or civilian injuries.
SAFER grants, which subsidized firefighter salaries, failed to reduce firefighter deaths, firefighter injuries, civilian deaths, or civilian injuries.
Without receiving fire grants, comparison fire departments were just as successful at preventing fire casualties as grant-funded fire departments.
Background
Almost all fire protection services in the United States are provided by local governments using unpaid volunteer firefighters.[4] America Burning, a 1973 report by the National Commission on Fire Prevention and Control, recognized that "[f]ire prevention, fire suppression, and public education on fire safety should remain primarily responsibilities of local governments, where familiarity exists with local conditions and the people being served."[5] However, America Burning did call on the federal government to produce research and analysis on the nation's fire problem, create a National Fire Academy to improve fire service training, and provide block grants for public education, local fire planning, and statewide information systems.[6]
One positive outcome of America Burning is the improved collection and dissemination of data on the incidents of fires and fire-related casualties. For almost 30 years, firefighter deaths have steadily declined. Chart 1 presents the number of firefighter deaths per 1 million emergency calls from 1981 to 2007.[7] In 1981, there were 7.83 firefighter deaths per 1 million calls. By 2007, the rate of firefighter deaths had plummeted to 1.70 per 1 million calls-- a reduction of 78.3 percent.
According to FEMA, 44 percent of firefighter deaths are the result of heart attacks, while trauma, including internal and head injuries, accounts for 27 percent of deaths.[8] Asphyxia and burns collectively account for 20 percent of deaths.[9]
During this same period, firefighter injuries also steadily declined. Chart 2 presents the number of firefighter injuries per 1,000 emergency calls from 1981 to 2007.[10] In 1981, there were 7.28 firefighter injuries per 1,000 calls. By 2007, the rate of firefighter injuries had declined to 2.12 per 1,000 calls--a reduction of 70.8 percent.
Accompanying the decline in firefighter casualties, the nation experienced a decline in the number of civilian deaths due to residential fires. Chart 3 presents the number of residential fire deaths per 100,000 residents from 1981 to 2005.[11] In 1981, there were 2.16 residential fire deaths per 100,000 residents. By 2005, the rate of residential fire deaths had declined to 0.95 deaths per 100,000 residents--a reduction of 56.0 percent.
For almost 30 years, America has benefited from improving fire safety. The declining trend in fire hazards is likely related to several factors, including improved firefighting services, fire code regulations, and building construction and engineering.
Despite the decades-long trend of increasing fire safety, Congress passed the FY 2001 National Defense Authorization Act, which established the AFG program to subsidize the routine activities of local fire departments and EMS organizations.[12] As detailed in the time-series analysis of injury and death trends in Appendix A, the creation of the fire grants does not appear to have substantially affected the national trends in firefighter deaths, firefighter injuries, and residential fire fatalities. In particular, the trend after the implementation of the fire grants (post-2001 segment) actually shows a lower decline in the rate of firefighter deaths, firefighter injuries, and residential fire deaths than the trend before the program was implemented (pre-2001 segment). Some suggest that this leveling out of fire casualties in recent years indicates that the nation "may have, in aggregate, reached a point of declining marginal returns with its current policies."[13] (For a technical discussion of the methodology, see Appendix A.)
Policymakers in the executive and legislative branches have had high expectations for the effectiveness of the fire grant programs. In 2008, Greg Cade, Administrator of the U.S. Fire Administration, stated that fire grants "help to ensure the nation's firefighters have the basic tools and resources necessary to safely perform their responsibilities, and therefore ultimately save lives and continue to protect all residents from fire."[14] More recently, Dennis Hunsinger, Acting Administrator for FEMA Region 10, proclaimed that the "Assistance to Firefighter Grant Program represents a major effort by the federal government to ensure that the nation's firefighters continue to have the basic capability they need to do their jobs, improve safety, and save lives."[15]
Members of Congress have also stated that fire grants to communities will save lives. In 2008, Representative Mike Conway (R-TX) claimed that the "Assistance to Firefighters Grant Program helps to ensure that firefighters in Midland have the tools they need to do their jobs, improve safety and save lives."[16] Commenting on the potential of a fire grant awarded to his district, Representative Joe Courtney (D-CT) said, "This grant will help the Town of Somers [Volunteer Fire Department] to perform their jobs more effectively, which will help to save lives and prevent injury."[17] These examples are just a small sample of the numerous press releases and newspaper stories quoting policymakers on fire grants awards.
The U.S. Fire Administration administers several types of fire grants. The AFG grants are intended to assist the firefighting and emergency response needs of fire departments and EMS organizations. The FP&S grants fund projects to improve the safety of firefighters and the public from fire and related hazards. These grants are intended to target high-risk populations and alleviate high incidences of fire casualties. The AFG and FP&S grants fund single-year projects.
SAFER grants were created in late 2003 to increase staffing levels by funding the salaries of career firefighters or by paying for recruitment and retention activities for volunteer fire departments. SAFER grants that are used to pay for the salaries of career firefighters require local matching contributions. Local matches are not required for grants that are used for recruitment and retention activities by volunteer fire departments.[18] SAFER grants that subsidize firefighter salaries can have lifespans of up to four years, but the total federal contribution for each funded position is capped at $100,000.[19] In addition, the federal contribution to firefighter salaries cannot exceed 90 percent in the first year, 80 percent in the second year, 50 percent in the third year, and 30 percent in the fourth year.[20] However, the American Recovery and Reinvestment Act of 2009[21] eliminated the local matching requirement for SAFER grants for firefighter salaries awarded in FY 2009 and FY 2010.[22] Grantees are required to retain SAFER-funded firefighters one year after the expiration of the grants.
From FY 2001 to FY 2009, Congress appropriated more than $5.7 billion in funding for fire grants.[23] (See Chart 4.) In addition to annual appropriation legislation, the American Recovery and Reinvestment Act appropriated $210 million for Fire Station Construction Grants for the renovation and construction of fire stations.[24]
Fire departments and EMS organizations from across the nation can apply for fire grant funding. By statute, fire grant award decisions are based on the merits of the applications and the needs of the community.[25] In addition to the merit-based system, the grants are also distributed by type of organization (career, volunteer, or combination), geographic location, and community type (for example, urban, suburban, and rural).[26]
A 2003 report by the U.S. Department of Agriculture (USDA) concluded that the "USFA Grant to Firefighters program was highly effective in improving the readiness and capabilities of firefighters across the nation."[27] However, the USDA report failed to utilize statistical and evaluation methods to assess the effectiveness of the AFG program in preventing deaths and injuries resulting from fires. The methodology used in this report, by comparison, includes control variables and allows for the inclusion of many cases in order to test competing hypotheses.
A 2007 report by the National Academy of Public Administration (NAPA) questioned the justification for creating the fire grants. NAPA noted:
The rationale for federal investment in basic firefighting and emergency medical services (EMS)--traditionally functions supported almost entirely by local communities--is not immediately clear. Moreover, even a federal program that has spent well over one-half billion dollars annually and has awarded grants to thousands of local organizations represents such a small fraction of State and local budgets that the impact on outcomes of interest to the public and Congress is likely to be modest.[28]
The NAPA report further acknowledged that the fire grant program "currently gathers little data from which to draw conclusions about which uses of its funds have the largest influence on public health and safety or other outcomes."[29] Complicating matters, FEMA offers little guidance to help grantees determine whether the grants are responsible for successful outcomes.[30]
Neither do fire grants appear to fulfill a homeland security function. The NAPA report acknowledges: "Basic fire incidents are usually well-handled in the U.S. and have been for some time, whereas large-scale, complex incidents are less well addressed and usually require cooperation of organizations and across jurisdictions."[31] However, the fire grant program "mainly funds local entities and isolated projects not tied to improving regional capabilities."[32]
Data and Modeling
Assessing a program's impact normally requires comparing the conditions of targets that have received an intervention with an equivalent set of targets that have not experienced the intervention.[33] The manner of selecting targets for inclusion in intervention and control groups can complicate the assessment of a program's real impact. For instance, when participation in a program is voluntary, the participants may be more likely to produce the desired effect regardless of whether they receive the intervention. In the fire grant program, fire departments that applied for grants may already have been more open to adopting innovative firefighting strategies than other fire departments that did not apply for grants. Thus, the evaluation methodology of this paper needs to address this dilemma.
Ideally, the most appropriate impact evaluation of the fire grant program would be an ex ante experimental design, in which grant funding is randomly assigned to intervention and control fire departments. Since the award of fire grant funding is based on a grant application process, researchers are left with ex post non-experimental designs for evaluating the impact of the grants on fire casualties. The techniques available for separating the effect of extraneous factors from the net effect of the fire grants are severely limited. Thus, this analysis uses repeated reflexive controls in which fire departments that received grants and fire departments that did not receive grant funding are observed repeatedly over time.
To institute reflexive controls, this paper uses panel data analysis. Panel studies observe multiple units over several periods. For this evaluation, the multiple units are individual fire departments. The addition of multiple data collection points gives the results of panel studies substantially more credibility than studies that make only two measurements, one before the intervention and one after.[34]
By increasing the number of data points compared to cross-sectional and time-series analyses, panel analysis increases the degrees of freedom and reduces possible collinearity among the independent variables, thus improving the efficiency of the econometric estimates. Further, the longitudinal nature of the panel data allows evaluators to analyze important policy questions that cross-sectional and time-series datasets cannot address. Panel analysis also reduces omitted variable bias by introducing cross-sectional and time-specific fixed effects into the model specification.[35]
Data.The data used in the evaluation were obtained from various sources. Data related to fire casualties and fire-related emergency incidents reported by fire departments were obtained from the National Fire Incident Reporting Systems (NFIRS).[36] NFIRS is an incident-based database managed by the U.S. Fire Administration and contains information on fire incidents voluntarily reported by fire departments, including information on fire casualties.
The fire grant award data were obtained from FEMA through repeated Freedom of Information Act (FOIA) requests.[37] In addition to the NFIRS and fire grant award datasets, socioeconomic data were obtained for the counties where the fire departments reside. The county-level demographic data included demographic population percentages broken down by race, gender, and age. These data were obtained from the Centers for Disease Control and Prevention (CDC).[38] The county-level economic data for income per capita and unemployment rates were obtained from the Bureau of Economic Analysis and the Bureau of Labor Statistics, respectively.[39]
The dataset contains data for 10,033 fire departments for an eight-year period (1999-2006). Of these fire departments, 5,859 (58.4 percent) received fire grant awards and 4,174 (41.6 percent) did not. Because NFIRS participation is voluntary, many fire departments do not report NFIRS data for all of the eight years in this dataset. Thus, the dataset is an unbalanced panel dataset: Some fire departments are observed for more years than other fire departments. The average fire department is observed for 4.8 years, with grant-funded fire departments averaging 5.2 years and comparison fire departments averaging 4.4 years.
Dependent Variables. Four different measures of fire casualties are used as the dependent variables: firefighter deaths,[40] firefighter injuries, civilian deaths, and civilian injuries. These dependent variables are expressed as the number of casualties per 1,000 residents in a given year based on U.S. Census place population estimates.
Explanatory Variables. The fire casualty variables not used as dependent variables in a particular regression model are used as explanatory variables. For example, in the regression models used to predict the rate of firefighter deaths, firefighter injuries, civilian deaths, and civilian injury rates are included as explanatory variables. These explanatory variables are thought to control for the level of risk that fire departments face each year.
The focus of this evaluation is to estimate the impact of fire grants on fire casualties. The AFG, FP&S, and SAFER grants are assumed to reduce or prevent fire casualties. The AFG grants are intended to assist the firefighting and emergency response needs of fire departments. AFG grants can be used to obtain equipment, protective gear, emergency vehicles, training, and other resources to protect firefighters and the public from fire and related hazards.[41] The grants are divided into three categories: general grants for equipment and protective gear, grants for the purchase of fitness equipment, and grants for the purchase of vehicles.
The FP&S grants fund projects to improve the safety of firefighters and the public from fire and related hazards.[42] These grants are intended to target high-risk populations and alleviate high incidences of fire casualties.[43] For example, the Greensboro Fire Department in North Carolina used an FP&S grant to paint a commuter bus with fire prevention and safety messages.[44] SAFER grants are intended to increase staffing levels by funding the salaries of career firefighters or paying for recruitment activities for volunteer fire departments.[45]
All of the grant variables are lagged one year and expressed on a per-capita basis using U.S. Census place population estimates. These variables are also expressed in 2006 dollars. The one-year lags of the grant variables allow for the purchase and operation of the equipment or staffing. In separate regressions, the grants, except for the SAFER grants, were also lagged for additional two-year and three-year periods to estimate the lasting effect of the technology, equipment, training, and other activities funded through the grants. The SAFER grants, which were created in 2003 and received the first appropriation in FY 2005, are assumed to have been first implemented in 2006. Due to the absence of publicly available NFIRS data for years beyond 2006, this evaluation can only assess the impact of the first year of SAFER grant implementation.
Fire departments are called upon to respond to many types of emergencies. To control for the types of emergencies addressed by fire departments, the analysis controls for the percentage of responses to fires, hazardous conditions, service calls, and good intent calls.[46] Fire incidents include fires occurring in buildings, structures, vehicles, other mobile properties, and the outdoors. Hazardous condition incidents include combustible or flammable spills or leaks; chemical, toxic, or radioactive conditions; electrical problems; biological hazards; and explosive bomb removal that does not involve fires. Service calls include incidents involving persons in distress, water problems, smoke or odor problems, animal problems or rescues, and public service assistance. Good intent calls include cancelled calls, events in which no emergency was found, and other mistaken calls. For the evaluation, fire incidents, hazardous conditions, and service calls are specified in the regression modeling, with good intent calls as the comparison. In addition, the total number of emergency incidents per 1,000 residents is included as an explanatory variable.
The socioeconomic variables are on the county level. The demographic variables include the American Indian, Asian, and black county population percentages, with the white population percentage as the comparison. To account for age patterns, the analysis includes explanatory variables for 20- to 29-year-olds, 40- to 59-year-olds, and the 60-year-old-and-older county population percentages, with the 19-year-old-and-younger population as the comparison. An explanatory variable for the percentage of the county population that is female is also included. Finally, the economic explanatory variables are the unemployment rate and income per capita (in 2006 dollars) for counties.
Table 1 presents the summary statistics for all of the fire departments in the dataset from 1999 to 2006. The first noticeable pattern is that fire casualties, while unfortunate, do not occur at high rates. On average, fire departments experienced 0.001 firefighter deaths per 1,000 residents and 0.052 firefighter injuries per 1,000 residents. For civilians, the rates are 0.119 deaths and 0.147 injuries per 1,000 residents. About 62 percent of emergencies reported by fire departments are fire-related incidents.
Table 2 provides summary statistics for fire departments that received fire grants and comparison fire departments from 1999 to 2006. Non-funded fire departments and grant-funded fire departments had the same rate of firefighter deaths per 1,000 residents. On average, both groups had 0.001 firefighter deaths per 1,000 residents. However, grant-funded fire departments had a higher firefighter injury rate. Grant-funded fire departments averaged 0.108 firefighter injuries per 1,000 residents, compared to 0.058 firefighter injuries per 1,000 residents for comparison fire departments.
Grant-funded fire departments had lower rates of civilian deaths and injuries than comparison fire departments. On average, grant-funded fire departments had 0.048 civilian deaths per 1,000 residents, while comparison departments had 0.137 civilian deaths per 1,000 residents. Grant-funded fire departments averaged 0.125 civilian injuries per 1,000 residents, compared to 0.183 civilian injuries per 1,000 residents for comparison fire departments.
Comparison fire departments reported a slightly higher percentage (64.7 percent) of emergencies that were fire-related incidents compared to grant-funded fire departments (61.1 percent). Grant-funded fire departments had a higher rate of emergency incidents than the comparison fire departments--39.2 incidents per 1,000 residents compared to 30.1 incidents per 1,000 residents. The grant-funded fire departments were more likely to be career or mostly career agencies than the comparison fire departments--23.8 percent compared to 12.7 percent.
Grant-funded and comparison fire departments are located in counties with approximately the same income per capita and unemployment rates. Further, grant-funded fire departments were located in cities and towns that were slightly more populated than the cities and towns of the comparison fire departments. For the other demographic variables, the differences between funded and non-funded fire departments are relatively small.
Modeling.All models were estimated using ordinary least squares (OLS) regressions. In addition, the models control for yearly fixed effects and cross-sectional fixed effects (individual differences related to each fire department), which account for time-invariant unobserved factors related to fire casualties in a particular jurisdiction that differ from fire casualties in other jurisdictions. The fixed-effects model helps to control for differences in fire casualties across jurisdictions that are not explained by the explanatory variables. Further, the fixed-effects model uses time-specific fixed effects, which involve the inclusion of year dummy variables. The year dummy variables control for unobserved factors that affect the nationwide occurrence of fire casualties and that are not accounted for by the control variables.
Including the specific differences attributable to each jurisdiction in the fixed-effects model helps to control for possible selection biasin the fire grant program's allocation of grants. Selection bias may occur if more innovative and effective fire departments are more likely than other departments to apply for and receive fire grants. The cross-sectional fixed-effects model helps to control for selection bias by giving each fire department an intercept, which allows the time-invariant individual differences of the fire departments to be absorbed.
The fixed-effects model can reduce selection bias, but it may not eliminate it entirely. If grant-funded fire departments are more innovative than non-funded fire departments, then the regression coefficients measuring the impact of fire grants will be biased downward, thus overstating the preventive effect of the grants on fire casualties.
Fire Casualty Models.The analysis of this evaluation concentrates on finding evidence of fire grants affecting fire casualties. Do fire grants reduce or prevent fire casualties? This is a reasonable research question to ask because the fire grant program has concentrated mainly on developing the capabilities of fire departments to react to fire emergencies.[47] By subsidizing the routine operations of fire departments, fire grants are thought to assist fire departments in becoming more proficient at fighting fires and providing emergency services. Thus, the improved operational proficiency of grant-funded fire departments should reduce fire casualties. The model specifications for how the fire grants and the other explanatory variables are thought to influence fire casualties are presented in Equations 1 through 4 below.
One variable missing from this evaluation is local fire department expenditures. Omitting local fire department expenditures from the regression modeling may negatively bias the results for the fire grant coefficients. While use of cross-sectional fixed-effects controls for time-invariant unobserved factors that differ among the individual fire departments, the omission of local fire department expenditures may still cause the evaluation to overstate the preventive effects of fire grants.
Several sets of regression models are estimated. The first set of regressions examines the impact of fire grants on all fire departments. The second set of regression models estimates the impact of grants in career and volunteer fire departments. This distinction is important because most volunteer fire departments are smaller suburban or rural departments, compared to professionally staffed fire departments, which are usually in larger, more urban communities.[48] The third set of regressions, presented in Appendix B, estimates the long-term impact of the grants on fire casualties by lagging most of the fire grant variables for additional two-year and three-year periods. Because SAFER grants were first awarded in FY 2005, these grants are only lagged for one year. The additional lags of the grant variables are done to estimate the lasting effect of the technology, equipment, training, and other activities funded through the grants.
Findings
A common theme emerges from the panel regression analysis. The fire grants have no statistically measurable impact on preventing fire casualties. Without receiving fire grants, comparison fire departments were just as successful at preventing fire casualties as grant-funded fire departments.
Did fire grants lagged one year reduce fire casualties? Table 3 presents the AFG grant findings from Models 1 through 4. The panel regression analysis is based on 10,033 fire departments that are observed for an average of 4.8 years. All of the standard errors reported in the following regression tables are robust to heteroskedasticity and autocorrelation.[49] In addition, the regressions are weighted by Census place population estimates.
For Model 1, the dependent variable is the firefighter death rate. Controlling for the other explanatory variables, none of the coefficients for the AFG, FP&S, and SAFER grant variables has statistically significant associations with firefighter deaths per 1,000 residents. The lack of statistical significance for the fire grant coefficients strongly indicates that these grants have no effect on preventing firefighter deaths. The income per capita of the county where the fire departments reside is negatively associated with the firefighter death rate. While this association is small, it is statistically significant (p =0.019). As will be seen with the other models that estimate firefighter deaths, finding explanatory variables that have statistically significant associations with firefighter deaths was difficult.
For Model 2, the dependent variable is the number of firefighter injuries per 1,000 residents. Controlling for the other explanatory variables, all of the coefficients for the fire grant variables fail to have statistically significant relationships with the firefighter injury rate. The lack of statistically significant results for the fire grants strongly indicates that these grants are ineffective at preventing firefighter injuries.
Other explanatory variables had statistically significant associations with firefighter injures. For each additional civilian death per 1,000 residents, firefighter injuries increased by 0.059 injuries per 1,000 residents. This association is statistically significant (p = 0.014). The association between civilian deaths and firefighter injuries is not surprising. Fire-related incidents that result in civilian deaths would typically occur under circumstances that are more dangerous to firefighters than incidents that do not result in civilian deaths.
The total number of emergency incidents per 1,000 residents is positively associated with the firefighter injury rate. Each additional emergency incident per 1,000 residents is associated with 0.0001 additional firefighter injuries per 1,000 residents. This association is statistically significant (p = 0.033).
The racial composition of counties does appear to influence firefighter injuries. A 1 percent increase in a county's black population is associated with an increase of 0.006 additional firefighter injuries per 1,000 residents. This association is statistically significant (p = 0.003).
The age and gender compositions of counties appear to be predictors of firefighter injuries. Compared to county population aged 19 or less, a 1 percent increase in the 20- to 39-year-old population is associated with a 0.013 decrease in firefighter injuries per 1,000 residents. This association is highly statistically significant (p = 0.001). A 1 percent increase in the 40- to 59-year-old population is associated with a decrease of 0.021 firefighter injuries per 1,000 residents. This finding is statistically significant (p = 0.008). For gender, a 1 percent increase in the female population is associated with a decrease of 0.023 firefighter injuries per 1,000 residents. This association is statistically significant (p = 0.018).
For Model 3, the dependent variable is the number of civilian deaths per 1,000 residents. The coefficients for the fire grant variables are statistically insignificant. The lack of statistically significant results means that the impact of grants on civilian deaths is statistically indistinguishable from zero.
On the other hand, other explanatory variables had statistically significant associations with the civilian death rate. For each additional firefighter injury per 1,000 residents, civilian deaths increased by 0.018 incidents per 1,000 residents. This association is statistically significant (p = 0.02). For each additional civilian injury per 1,000 residents, civilian deaths increased by 0.032 incidents per 1,000 residents. This association is statistically significant (p = 0.008).
County income per capita and unemployment rates appear to have no association with the civilian death rate. The type of emergency incidents and emergency incident rate also appear to have no association with civilian deaths.
However, two of the county demographic variables had statistically significant associations with civilian deaths. Compared to a county's white population, a 1 percent increase in a county's Asian population is associated with a 0.003 increase in civilian deaths per 1,000 residents. This association is statistically significant (p = 0.004). Compared to county population aged 19 or younger, a 1 percent increase in the 20- to 39-year-old population is associated with a 0.002 decrease in civilian deaths per 1,000 residents. This association is statistically significant (p = 0.007). The other demographic variables appear to have no association with the civilian death rate.
For Model 4, the dependent variable is the number of civilian injuries per 1,000 residents. The coefficients for the fire grant variables are statistically insignificant. The lack of statistically significant results means that the impact of fire grants on civilian injuries is statistically indistinguishable from zero.
On the other hand, other explanatory variables had statistically significant associations with civilian injuries. For each additional firefighter injury per 1,000 residents, civilian injuries increased by 0.073 incidents per 1,000 residents. For each additional civilian death per 1,000 residents, civilian injuries increased by 1.69 injuries per 1,000 residents. Both of these associations are statistically significant (p = 0.015 and p = 0.019).
As with Model 3, county income per capita and unemployment rates appear to have no association with the civilian injury rate. The type of emergency incident appears to have no association with civilian injuries. However, each additional emergency incident reported by fire departments per 1,000 residents is associated with a 0.0004 increase in civilian injuries per 1,000 residents. This association is statistically significant (p = 0.006).
Only two of the county-level demographic variables are associated with civilian injuries. Compared to the percentage of the county population that is 19 or younger, a 1 percent increase in the 20- to 39-year-old population is associated with a 0.008 decrease in civilian injuries per 1,000 residents. In addition, a 1 percent increase in the 40- to 59-year-old county population is associated with a 0.02 decrease in the civilian injury rate. These associations are statistically significant (p = 0.047 and p = 0.037).
Do fire grants reduce fire casualties reported by career fire departments? For career or mostly career fire departments, Table 4 presents the fire grant findings from Models 5 through 8. The panel regression analysis is based on 1,644 career or mostly career fire departments that are observed for an average of 5.8 years. None of the fire grant coefficients yield statistically significant results. These findings suggest that fire grants awarded to career and mostly career fire departments have no measurable impact on reducing fire casualties. For Model 5, the attempt to estimate the rate of firefighter deaths failed to yield statistically significant coefficients for the explanatory variables.
Do fire grants reduce fire casualties reported by volunteer fire departments?For volunteer and mostly volunteer fire departments, Table 5 presents the fire grant findings from Models 9 through 12. The panel regression analysis is based on 8,389 volunteer or mostly volunteer fire departments that are observed for an average of 4.7 years. None of the coefficients for the fire grant variables have statistically significant associations with the fire casualties. For Model 9, the attempt to estimate the rate of firefighter deaths failed to yield statistically significant coefficients for the explanatory variables.
Do fire grants lagged one to three years reduce fire casualties?Tables B-1 through B-2 in the Appendix present the findings for the regressions when the fire grants are lagged an additional two and three years. Due to space limitations, these tables present only the coefficients for the fire grant variables. These model specifications test if the fire grants have additional impact on fire casualties beyond their first year of implementation. Table B-1 presents the findings for all types of fire departments. None of the fire grant variables have statistically significant associations with fire casualties.
Table B-2 presents the findings for career and mostly career fire departments. Table B-3 presents the findings for volunteer and mostly volunteer fire departments. None of the fire grant variables have statistically significant associations with fire casualties.
Discussion
The strength of this evaluation's methodology resides in its use of panel data to compare fire departments that received grants to fire departments that did not receive grants. In addition, the evaluation compares the impact of the grants before and after grant-funded fire departments received federal assistance. The evaluation analyzed more than 10,000 fire departments from across the nation from 1999 to 2006. Fire grants appear to be ineffective at reducing fire casualties. Without receiving fire grants, comparison fire departments were just as successful at preventing fire casualties as grant-funded fire departments. This finding held when the analysis is limited to career and mostly career fire departments and to volunteer or mostly volunteer fire departments.
The findings of this evaluation were foreshadowed when the 2007 NAPA report concluded that the "program's strategy of improving firefighting response capabilities, however effective it is at doing this, may not represent the most cost-effective way to reduce either public or firefighter deaths and injuries."[50] In addition, the NAPA report noted:
One argument that has been made forcefully by experts on the fire problem over the last four decades is that dollars used to reduce the number of fire incidents are likely to have greater impact on fire safety relative to their cost than dollars used to improve response to fires when they break out.[51]
Another factor that may prevent the fire grants from being effective is the issue of supplanting. Supplanting occurs when federal funds are used to replace local funds, such as when federal funds intended for purchasing fire protection equipment are instead used to pay for items that grantees would have purchased even if they did not receive federal funding. Similar to fire grants, federal grants to police departments have been used for supplanting.[52]
The NAPA report acknowledges that "[t]here is no conclusive data available to analyze whether federal funds...are being used to supplant local funds that would have been spent on the same items. However, anecdotal evidence abounds on this topic."[53] While this report did not attempt to assess the occurrence of supplanting, the inability of the fire grants to reduce fire casualties may have resulted from supplanting. If supplanting frequently occurs with fire grants, then federal assistance will not lead to a net increase in the provision of protective fire services.
As currently implemented, fire grants do not appear to fulfill a federal homeland security function because they are too focused on subsidizing the routine operations of basic fire services. Federal homeland security grants should be focused on building critical capabilities, which includes purchasing equipment, training first responders, and performing regional exercises to test competency.[54] Federal assistance should supplement, not supplant, state and local responsibilities. In other words, federal assistance should perform value-added functions and not replace or supplant the normal responsibilities of state and local governments.
By subsidizing firefighter salaries, the SAFER grants supplant rather than supplement state and local responsibilities. In addition, the AFG grants are routinely used to purchase vehicles and equipment used for routine activities, such as pumpers, tankers, self-contained breathing apparatuses, and Personal Alert Safety Systems. While these items are important to providing basic fire services, federal funding of these items does not supplement or add to the capabilities of local fire departments to perform homeland security tasks. The federal funding does not perform a value-added function because it replaces local responsibilities. Yet federal assistance for the purchase of interoperable communication equipment and training to help local fire departments from different jurisdictions to coordinate responses to large-scale catastrophic incidents, such as natural disasters and acts of terrorism, is a more appropriate use of federal resources.
Conclusion
From FY 2001 to FY 2009, Congress appropriated $5.7 billion in funding for fire grants.[55] Grant award data were matched to the National Fire Incident Reporting System, an incident-based database of fire-related emergencies reported by fire departments. In addition, the grant data were matched to data from the Census Bureau, Bureau of Economic Analysis, and Bureau of Labor Statistics. Using panel data from 1999 to 2006 for more than 10,000 fire departments, this evaluation uses fixed-effects regressions to estimate the impact of fire grants on four different measures of fire casualties: firefighter deaths, firefighter injuries, civilian deaths, and civilian injuries.
This impact evaluation finds that fire grants, including grants that subsidize the salaries of firefighters, had no impact on fire casualties:
AFG grants used to purchase firefighting equipment, vehicles, and fitness equipment failed to reduce firefighter deaths, firefighter injuries, civilian deaths, or civilian injuries.
FP&S grants that funded fire prevention and safety projects failed to reduce firefighter deaths, firefighter injuries, civilian deaths, or civilian injuries.
SAFER grants that subsidized firefighter salaries failed to reduce firefighter deaths, firefighter injuries, civilian deaths, or civilian injuries.
Compared to fire departments that did not receive fire grants, grant-funded fire departments were not more successful at preventing fire casualties.
David B. Muhlhausen, Ph.D., is Senior Policy Analyst in the Center for Data Analysis at The Heritage Foundation.
Appendix A:
Did the Fire Grant Program Produce Structural Changes in Fire Death and Injury Trends?
Karen A. Campbell, Ph.D.
While the long-term trend in firefighter death, firefighter injury, and residential fire death rates presented in Charts 1-3 has been downward, the implementation of the fire grant program may have caused shifts in these trends, indicating a policy impact. To identify any statistically significant shifts in the trends that may not be obvious from the graph, formal structural break tests were performed on the three time series. The results lend empirical support to the hypotheses that:
- Fire grant funding did not decrease the trend reductions in these injury and death rates and
- The factors driving these rates down had their largest impacts in the previous decade, and additional policy interventions will likely produce only small marginal benefits in terms of lower injury and death rates.
Two tests were used to check for structural breaks in the data series. Both tests rely on fitting linear trends in order to minimize the sum of squared residuals. In each of the univariate series, the ordinary least square (OLS) regression was:
where β is a constant, δ is the slope parameter, and j specifies the break points. For each break point, the parameters are estimated by minimizing the sum of squared residuals, u.
The first test did not specify when the structural break occurred. Instead, it relied on the Bai and Perron method[56] to search for the likeliest time that a structural break occurred. This procedure searches through each point in time and compares the total sum of squared residuals from the OLS estimate on each segment of the break to all other points in time. The point with the lowest sum of squares is identified as the best break point.
The tests showed that the best[57] structural break points occurred in 1993 in the firefighter death rate data, 1995 in the firefighter injury data, and 1997 in the residential fire death rate. This suggests that fire grants created in 2001 did not significantly impact the trajectory or levels of these rates--in the sense of creating a significant structural break-- beyond the impact of previous factors.
The second test, a Chow test,[58] gives further insight into the data around the implementation of the program. This test specified a break point at 2001 and tested the hypothesis of whether the parameters of the data were the same pre-2001 and post-2001.[59]
The Chow test assumes that the distribution of the time series variable is not changed by the policy causing a structural break in the data because a break only affects the parameters (slope and intercept) of the time series. This means that a policy or factor that caused the time series to shift or grow at a different rate did not change the random component (variance) of the time series variable.[60]
Testing of the pre-2000 and post-2001 data found a significant difference between the two segments. However, the coefficient difference between the two segments reveals that the post-2001 segment actually shows a slower rate of decline in the firefighter deaths, firefighter injuries, and residential fire deaths compared to the pre-2001 segment. That is the steep declines in death and injury rates prior to 2001 slowed down significantly in the post-2001 segment.
While causal inference cannot be made by this analysis, it does empirically show that the fire grant did not further reduce the already declining death and injury rates. To the extent that the slowdown in death and injury rates was occurring due to normal diminishing returns to the true causes,[61] this further supports the hypothesis that additional grant funding will not be as effective in reducing the death and injury rates.
Karen A. Campbell, Ph.D., is Policy Analyst in Macroeconomics in the Center for Data Analysis at The Heritage Foundation.
