Last month, congressional negotiators reached a bipartisan deal to reauthorize Department of Labor (DOL) job-training programs. Overall, the compromised legislation, the Workforce Innovation and Opportunity Act (WIOA), continues the funding of federal job-training programs that have a long history of failure based upon the results of large-scale experimental evaluations.[1]

Despite continuing job-training programs that have never been scientifically demonstrated to work, WIOA eliminates 15 minor job-training programs—far short of the 35 programs consolidated or eliminated in the House-passed Supporting Knowledge and Investing in Lifelong Skills Act.

Finding Out What Works

Federal job-training programs, as well as many other federal social programs, are seldom evaluated to determine whether they are actually accomplishing their intended purposes. As part of its obligation to spend taxpayers’ dollars wisely, Congress should expand the evaluation provisions in WIOA to include the job-training programs that have yet to be evaluated.

The WIOA legislation contains some good provisions regarding the experimental evaluation of programs. For example, WIOA requires the DOL to conduct a single multisite impact evaluation of federal job-training programs using random assignment—the most scientifically rigorous method for assessing impact. Based on historical precedent, this single multisite impact evaluation mandate will likely mean that one or two large funding streams, such as the adult and dislocated worker training programs, will be evaluated.

However, this limited focus means that the rest of the other authorized training programs will not undergo multisite experimental evaluations. Multisite evaluations provide the most reliable information about program effectiveness. Often factors unique to an area can cause a program to succeed or fail there, but these findings change when researchers evaluate the program across a large number of sites. Congress should require experimental multisite evaluations if it wants to know how effectively job-training programs help disadvantaged workers.

The following programs and funding streams are not likely to undergo multisite experimental impact evaluations:

• Job Corps,
• Wagner–Peyser Act Employment Services,
• Native American programs,
• Migrant and Seasonal Farmworkers programs,
• National Dislocated Worker Grants,
• YouthBuild,
• Adult Education and Literacy Act programs, and
• Rehabilitation Act of 1973 programs.

Only one of these programs or funding streams has undergone a multisite experimental evaluation: Job Corps. About 13 years have passed since the results of the National Job Corps Study were published in 2001. While the results strongly indicated that Job Corps was a failure,[2] Congress continues to provide ample funding for the program. The programs operated under the Wagner–Peyser Act Employment Service have been around since the New Deal, yet this program has never undergone a multisite experimental evaluation.

Congress should require the DOL to conduct multisite evaluations using randomized controlled trials of all the major job-training programs it operates to evaluate these programs’ effectiveness.

Sanctions for Not Conducting the Evaluations

The DOL has historically delayed conducting or publishing congressionally mandated evaluations of its job-training programs. These evaluations often find the programs ineffective. When Congress passed the Workforce Investment Act (WIA) in 1998, it required the DOL to complete an evaluation of the program’s effectiveness by 2005. The Bush Administration ignored this deadline and did not award the contract for conducting the randomized controlled trial until it was leaving office.

Further, the forthcoming evaluation assesses only the WIA adults and dislocated funding streams while failing to assess the effectiveness of the WIA youth funding stream. This is regrettable, because youth are a population that has been historically poorly served by federal job-training programs.[3] The final results may not be available until 2016—17 years after the passage of WIA.

To prevent this from happening, the evaluation should include sanctions on the DOL for non-compliance with the congressionally mandated deadline. This could take the form of automatic funding reductions, such as 5 percent sequestration of agency funds until the evaluation takes effect. Another possibility would include sequestering a portion of the pay of the Senior Executive Service and political employees with responsibility for conducting the evaluations. For example, their pay could get reduced by 5 percent for each year the evaluation is delayed beyond its congressionally mandated completion date (i.e., 5 percent the first year, 10 percent the second year, etc.). This would incentivize the bureaucracy to conduct the evaluation in a timely manner.

Congress should also include a requirement to brief congressional leaders on the findings by a certain date and then publicly release the evaluation within 90 days afterward. The DOL has a history of not releasing unfavorable evaluations for years. If the evaluations show the programs perform poorly, the DOL should promptly make that known.

Multisite Experimental Evaluations Are Critical

Some may argue that the performance-monitoring provisions in WIOA can substitute for multisite experimental evaluations. While the compromise legislation is intended to improve performance monitoring and accountability, these provisions have serious limitations.

First, the revised performance-monitoring system cannot inform policymakers about the true effectiveness of federal job-training programs because there are no control groups to make valid comparisons.[4]

Second, administrators of federal job-training programs have a history of manipulating performance data to make the programs appear to be more successful than they actually are.[5] For example, the Government Accountability Office found that local program administrators often decide to record in the performance-monitoring system only those trainees whose results help them meet their performance targets.[6] A randomized controlled trial prevents administrators from cherry-picking the trainees they evaluate.

Not Controversial

Finding out what works should not be a controversial issue. If the legislators pushing for passage of WIOA are genuinely interested in finding out what works, then they should support the addition of legislative provisions that would ensure that more federal job-training programs are successfully evaluated for effectiveness.[7]

—David B. Muhlhausen, PhD, is a Research Fellow for Empirical Policy Analysis and James Sherk is Senior Policy Analyst in Labor Economics in the Center for Data Analysis, of the Institute for Economic Freedom and Opportunity, at The Heritage Foundation.