As a shocking number of American children continue to perform poorly in the core subjects of mathematics and English, most Americans would be surprised to learn that their tax dollars are funding education frills developed by regional laboratories of the U.S. Department of Education. According to Diane Ravitch and Chester Finn, former Assistant Secretaries at the Department of Education's Office of Educational Research and Improvement (OERI), which oversees these laboratories, regional education labs are nothing more than
A collection--now ten--of smallish non-profit organizations, each located in a different region, that undertake a mishmash of research, dissemination, and technical assistance activities, aimed mostly at state and local education agencies.... The program as a whole has outlived whatever justification it once had.... The putative beneficiaries are schoolchildren who need to learn more. Yet the money actually goes to well-paid professionals functioning as middlemen, sitting in comfortable offices distant from the classroom, and devoting much of their energy to ensuring that their federal gravy train does not halt on the tracks.2
The federal government can play an important role in gathering information, conducting research, and disseminating valid findings. There may not be a "magic bullet" for every problem in America's failing schools, but educators have developed many effective teaching methods that could be replicated in classrooms across the country. Currently, OERI is in charge of this function. Its National Center for Education Statistics (NCES) has gathered, analyzed, and disseminated national data. But the remainder of OERI's work is fragmented and highly vulnerable to politicization and manipulation.
This is especially true with respect to OERI's regional education laboratories. These labs have been around for more than 32 years and have received over $750 million in federal funding. Yet, by nearly all accounts, what they produce has had little impact in revolutionizing education in America's schools.
To a great extent, reformers have been frustrated in their efforts to improve the labs' effectiveness: The influence of lobbyists who promote fanciful education projects has been too strong--and the resolve of Congress has been too weak--for managerial and substantive change to be implemented successfully.
Education research is best conducted by private and independent organizations that are funded based on their ability to translate research findings into successful methodology and practice. The sole determinant in deciding whether a program will be implemented in the schools should be an increase in academic scores, not how much the educational bureaucracy or lobbyists like the program. Yet the laboratories and the federal bureaucracy rarely analyze how well programs are working.
Members of Congress interested in streamlining the federal role in education should examine the research conducted in OERI laboratories and demand more accountability in return for the approximately $50 million the labs receive from the federal government every year. The best way to achieve this goal would be to allow the private sector, as well as other agencies, to compete for funds targeted to the work of these labs, and to base this funding on individual projects. Or Congress could simply roll over the labs' funding into a block grant to states, which then could set their own research priorities and hire whomever they see fit to conduct the work.
WHAT FEDERAL EDUCATION LABS DO
The largest portion of the federal government's role in education research is administered through four programs of the Office of Educational Research and Improvement, at a total cost of $510 million in fiscal year (FY) 1998. These programs include:
-
Regional Educational Laboratories (10);
-
National Education Research Institutes (5) in charge of 11 National Research Centers;
-
Educational Resources Information Centers (ERIC); and
-
Eisenhower National Clearinghouse for Mathematics and Science, and Regional Consortia.
Regional education laboratories were created in 1965 as part of President Lyndon B. Johnson's Great Society education initiative, the Elementary and Secondary Education Act (ESEA). The goal of the labs, according to the U.S. Department of Education, is to bring "structure and cohesion to a fragmented federal role in educational research and development."3 They are the Department's largest research and development investment. In 1996, Congress appropriated around $51 million for the labs, and this is supplemented by funds obtained from grants and contracts through the Department of Education and other federal, state, and local sources.4
Contracts to run the labs are awarded every five years. The labs report to regional boards comprised primarily of educators and, in many cases, the chief state school officers.5
A Snapshot of the Regional Labs6
-
The Appalachian Educational Laboratory (AEL) is based in Charleston, West Virginia, and serves Kentucky, Tennessee, Virginia, and West Virginia. One of the oldest labs, it specializes in rural education and houses the ERIC Clearinghouse on Rural Education and Small Schools. It promotes the integrity of rural small schools in a global economy. AEL received $4.1 million in FY 1996. It developed the well-known Questioning and Understanding to Improve Learning and Thinking (QUILT) teaching model and has studied the Kentucky Education Reform Act of 1990 to analyze the effects of large-scale changes in state policy on rural education. It also publishes Family Connections, a weekly guide for parents used in 45 states and available to more than 60,000
families.7 -
The Mid-Atlantic Laboratory for Student Success (LSS) is based in Philadelphia, Pennsylvania, and serves Delaware, the District of Columbia, Maryland, New Jersey, and Pennsylvania. It specializes in urban education and received $5.2 million in FY 1996. (An Eisenhower Regional Consortium also is located in Philadelphia.) LSS runs the Learning City Program, a broad school-family-community approach to promoting the educational attainment of urban youth. Studies show that students in the program achieve higher math and reading scores on standardized tests, as well as higher levels of self-esteem.8
-
The Mid-Continent Regional Educational Laboratory (McREL) is based in Aurora, Colorado, and serves Colorado, Kansas, Missouri, Nebraska, North Dakota, South Dakota, and Wyoming. It specializes in curriculum, training, and instruction and received $4.2 million in FY 1996. McREL has been heavily involved in developing national standards, It also has developed a system to modernize the South Dakota education system and has published The Systemic Identification and Articulation of Content Standards and Benchmarks.9
-
The North Central Regional Educational Laboratory (NCREL) is based in Oak Brook, Illinois, and serves Iowa, Illinois, Indiana, Michigan, Minnesota, Ohio, and Wisconsin. It specializes in technology education and received $6.6 million in FY 1996. This lab has studied, among other things, the Iowa legislature's early childhood programs to help target state funding.10 It also has developed 22 professional development programs and has transmitted them to more than 23,000 schools across the region.
-
The Northeast and Islands Regional Educational Laboratory (LAB) is located at Brown University in Providence, Rhode Island, and serves Connecticut, Maine, Massachusetts, New Hampshire, New York, Puerto Rico, Rhode Island, Vermont, and the Virgin Islands. It specializes in language and cultural diversity and received $6.1 million in FY 1996. (An Eisenhower Regional Consortium also is located in Cambridge, Massachusetts, at the Eisenhower Regional Alliance for Mathematics and Science Education Reform.) LAB is one of the newest labs, having taken over another lab that was caught overbilling the government. It currently is exploring strategies to implement academic standards for English-language learners and to identify math and science instructional practices that effectively address the needs of diverse learners.11
-
The Northwest Regional Educational Laboratory (NWREL) is based in Portland, Oregon, and serves Alaska, Idaho, Montana, Oregon, and Washington State. It specializes in school processes and received $5.2 million in FY 1996. An Eisenhower Regional Consortium is located in Portland as well. NWREL has developed Onward to Excellence, a school reform model used by 2,000 schools over the past 14 years, and a program on teaching students how to write well. It also has trained thousands of teachers and assists schools technically.12
-
The Pacific Region Educational Laboratory (PREL) is based in Honolulu, Hawaii, and serves American Samoa, Guam, Hawaii, the Northern Marianas Islands, the Marshall Islands, Micronesia, and Palau. It specializes in language and cultural diversity and received $3 million in FY 1996. PREL's major function is the training and networking of remote island teachers. It also attempts to integrate island culture into local education13 and holds the Pacific Education Conference, which draws about 1,200 participants each year. PREL has studied school finance, at-risk students, and cultural learning in the Pacific.14
-
The Southern Regional Vision for Education (SERVE) is based in Greensboro, North Carolina, and serves Alabama, Florida, Georgia, Mississippi, North Carolina, and South Carolina. It specializes in early childhood education and received $5.6 million in FY 1996. SERVE studies site-based management in North Carolina and dropout prevention in South Carolina. It also has helped develop a school board member training program in Mississippi and has conducted a study of school restructuring.
-
The Southwest Educational Development Laboratory (SEDL) is based in Austin, Texas, and serves Arkansas, Louisiana, New Mexico, Oklahoma, and Texas. Specializing in language and cultural diversity, it received $5.5 million in FY 1996. SEDL monitors a program to foster understanding and plan for the education of students on both sides of the border.15
-
The WestEd Laboratory (WestEd) is based in San Francisco, California, and serves Arizona, California, Nevada, and Utah. It specializes in assessment and accountability. WestEd received $5.5 million in FY 1996. Its principal research effort has been a study of situational teaching, a model now used by more than 4,800 teachers in California.16 The lab also engages in "whole school" reform, language and cultural diversity, state alliance projects, and policy support and studies programs. In addition to developing the Infant/Toddler Caregiver Training Program, it has adapted the Success for All reading program from the National Research Center at Johns Hopkins University to serve students with limited English proficiency more effectively. WestEd has helped more than 100 schools adopt and implement the Success for All model, which is touted by OERI as one of the top 16 lab projects, to meet local needs.
WHAT'S WRONG WITH FEDERAL EDUCATION LABS
Although the ten regional laboratories seem to be doing interesting work, their impact on education reform has been small; in some cases, their work may have undermined efforts at reform. Among the many problems associated with the work of these labs are the following:
- Evidence of Waste and Abuse. A March 1998 audit of WestEd by the U.S. Department of Education's own inspector general (IG) found that the lab did not comply with certain federal laws and regulations in managing its Regional Educational Lab contracts. The audit also found that WestEd's indirect cost estimates negotiated by the Department of Education do not reflect all of its indirect costs.17 Specifically:
WestEd leased space to a radio station and a computer facility in buildings purchased with federal funds for educational research purposes and retained all subsequent profits, totaling $627,000, from December 1994 to November 1996.
WestEd used "lease-purchase agreements that resulted in excessive charges for furniture, equipment, and building improvements. The interest portion of the lease-purchase payments was excessive since WestEd had funds available in its reserve to make cash purchases. WestEd also accelerated charges to the contract for the purchases and charged interest to the contract during periods when interest was an unallowable cost."
WestEd "improperly billed the contract for indirect costs on work performed by subcontractors."
It also "gave the impression that indirect costs remained fairly constant when in fact the indirect cost rate increased 29 percent over the past three years. For fiscal year 1996, WestEd's actual indirect cost rate was 45 percent rather than its stated rate of 12.8 percent."
The inspector general recommended that the department require WestEd to use federal funds and property efficiently and for their intended purposes. The IG also required WestEd to return approximately $131,000 in federal funds used for unallowable interest, improperly computed indirect costs, and other unallowable direct costs. In addition, the IG concluded that WestEd should use the $2.6 million of accumulated rental profits to reduce program objectives and expenses, leaving over $300,000 in rental profits available for these purposes annually in future years.18
Another audit, on February 25, 1998, dealt with the costs incurred by the University of North Carolina, Greensboro, and the Southeastern Regional Vision for Education under the U.S. Department of Education's Regional Educational Laboratories contract.19 This audit found that the university "could not provide assurance that approximately $2.3 million in salaries, fringe benefits, and related indirect costs between December 1, 1995 and November 30, 1996 were allocated in direct relationship to the time and effort made toward the Regional Educational Laboratories contract." The department also was unable to "make an informed decision regarding $415,000 of proposed subcontract actions because the University did not obtain cost information from prospective subcontractors as required by the Federal Acquisition Regulation." Though the university has taken action to correct itself, the inspector general recommended that the department require the university to submit additional information for evaluation and assurance of compliance with appropriate
federal regulations.
- Questionable Quality and Value of Research. As Peirce Hammond, director of the Office of Reform Assistance and Dissemination at OERI, explains, labs have to "strike a balance between the urgency of children's needs. The issue is what do you do with partial knowledge. Take a chance? Or take a risk? Labs are in the middle of this dilemma all the time."20 But a 1997 District and State Survey conducted by the U.S. Department of Education found that nearly half of the states and districts that had contact with regional labs found them to be of little or no help in understanding or implementing comprehensive standards-based reform.21 Their emphasis on speedy delivery is also not reassuring, because sound research methods often rely on experiments that take months, if not years, to conduct.
Lack of Objectivity. The labs consider themselves to be independent and objective and, through their national network, to possess a capacity to tackle nationwide needs and encourage efficiency. "By using proven practices and applying Laboratory expertise, educators and policymakers can avoid duplications, mistakes, and unnecessary costs of reinventing programs," boasts one lab manual.22
But even a quick review of lab Internet sites shows big gaps in the information made available. There are few details on free-market ideas or initiatives such as school choice, even though there are many recent studies on school choice programs and their positive appeal to inner-city parents (especially with the increase in privately sponsored programs). Somewhat more information is available on charter schools and the contracting-out of school services.
- Promotion of Fads. The labs are supposed to disseminate proven methods of achieving academic improvement, yet many of the methods they promote are widely criticized fads, such as whole language. Furthermore, the research they conduct often seems to rely more on anecdotes and theories than on facts.
For example, OERI lists AEL's Family Connections as one of the Regional Laboratories' top 16 tested ideas for teaching and learning. Yet the program was reviewed by a panel selected by the AEL that included Head Start coordinators, Title I directors, teachers, and Parent-Teacher Association leaders. The panel gave high marks to the program for "developmental appropriateness, interest, understandability, and usefulness to parents," but failed to mention any impact on students' academic achievement.23
AEL also touts its QUILT program, which was designed to "train teachers to use well-placed questions to stimulate students to think, question, and learn." QUILT "helps teachers create a classroom environment that is more reflective, more student centered, more inquiry based, and more metacognitive."24 Yet no one--not even the labs--can provide evidence that QUILT actually improves students' test scores. The data from a four-state field test indicate only that QUILT-trained teachers employed what they learned to change classroom practice and that students in their classrooms asked more questions that reflected "better thinking."25
An LSS project entitled "The Rural Cluster Case Study: Say Yes to a Youngster's Future" promotes a math and science curriculum "designed to be inquiry-based, hands-on, cooperative, [and] self-paced." There is, however, no mention of whether the program is or is not backed by research or solid evidence that demonstrates proven academic gains. LSS claims that the overall goal of this program is to test the feasibility of implementing the "Say Yes" program and to determine how to scale up implementation--with no mention of seeing how "Say Yes" improves academic achievement.26
The labs also frequently use terms like "feeling positive" about the classroom and school environment, "situational learning," and "better academic self-concept" to describe what they are researching. NCREL developed "Strategic Reading," an "innovative" approach that uses reading to improve instruction and encourages "team teaching, active roles for students, and meaningful principal participation," among other things.27 NWREL is developing processes and resources for schools to establish classroom environments that are developmentally and culturally appropriate for young children.
THE LABS' QUESTIONABLE RESEARCH
E. D. Hirsch, a professor at the University of Virginia, points out that terms such as "developmentally appropriate," "modern," "hands-on," "integrated," "interesting," and "individualized" are often used to describe teaching techniques that are ideologically loaded and neither grounded in nor backed by solid research that tests their effectiveness. Several studies show, for instance, that learning ability does not correlate with age and background--an idea that underpins "developmentally appropriate" programs. Juxtaposing developmentally appropriate methods with traditional methods of teaching students makes the "progressive" techniques more attractive, but little if any research supports their effectiveness.28
Labs have different methods of evaluation and place different degrees of importance on research. Yet all use the term "research-based" to describe the programs they promote, even though "[they] don't have the luxury of running controlled experiments," as Wesley Hoover, President and CEO of the Southwest Educational Development Laboratory, admits.29 At the same time, "Direct Instruction," a program which uses memorization and drill and has been proven to be effective in boosting the academic outcomes of disadvantaged students,30 is not widely distributed or prominently featured by the labs because it is not interesting to the teachers who have to use it. "[The] program will not work," claims Hoover, because "75 percent of teachers in our Elementary schools come from a whole language background. You need something else that `walks them through Direct Instruction.'"
Flaws in Lab Research
A June 1993 analysis of OERI and its labs by Maris Vinovskis, former Research Advisor to the Assistant Secretary at OERI and currently a professor at the Department of History and Institute for Social Research at the University of Michigan, provides a glimpse of the quality of research and development at five OERI Regional Education labs.31 Vinovskis praised the work of some of the labs, but he also found several problems:
-
The mission of the Northeast and Islands Regional Educational Laboratory (LAB) at Brown University is to "promote systemic school improvement in the Northeast and Islands region by supporting researchers who conduct collaborative inquiry alongside educational practitioners and community members." Yet LAB launched an intensive Collaborative Action Research effort in three sites without an adequate research design for a large-scale or long-term undertaking.32 LAB has put together a "nice series of briefing papers on teacher quality, pension portability, and teacher incentives," but the product only summarizes existing policy papers.33
-
The main emphasis of the Southwest Educational Development Laboratory (SEDL) is "ensuring educational equality for mostly minority or handicapped children and youth who live in poverty."34 One of SEDL's projects was an investigation of small and economically disadvantaged rural schools in five different states over several years. Each of the five demonstration schools, all of which had indicated an interest in improving student achievement scores, was visited monthly by an SEDL staff member responsible for facilitating change in that institution. An analysis of the plan found that because of a lack of standardized measures of educational success (and failure to include adequate controls in the research design), it was impossible to make any systematic comparisons among the schools. And since the schools studied were not necessarily typical of others in that region, it was difficult to know how much could be generalized from these findings. Because there were no control groups in the study, it was hard to establish which factors were instrumental in improving student achievement. The analysis found the SEDL project an interesting but very limited demonstration effort at five diverse sites, which does not lead to information that would be useful in determining key factors in improving rural education.35
-
The Mid-Continent Regional Educational Laboratory (McREL) emphasizes working "collaboratively with its clients to improve educational policy and practice through the application of knowledge from research, development, and experience."36 It has produced some useful policy papers, but many are little more than a catalog of research, without adequate attention given to the quality of the work or the diversity of material in the field.37 In addition, there are serious flaws in the way the lab tallied the results of a questionnaire it sent to local public school superintendents in the seven states it serves. Although the overall response rate was only 40 percent and varied among the states, the analysts simply grouped all of the returns together to get an overall regional profile.38
-
The signature program of the Northwest Regional Educational Laboratory (NWREL)--"Onward to Excellence" (OTE)--is referred to as a "research-based" school improvement process that focuses the attention and resources of school systems and their communities on learning success for all students and develops the capacity within school systems for continuous improvement through shared leadership.39 OTE was tested on nearly 2,000 schools, yet a study of its impact did not explain why certain OTE programs were chosen for the study and others were not. Moreover, an in-depth study of OTE programs could not explain why some of them succeeded while others failed. The study failed to look at non-OTE schools to see whether any of the improvements in the OTE schools were due in part to other general changes affecting all schools during those years. And because the term "success" was not very well defined, OTE programs that set low goals conceivably could be labeled as "successful" as--or even more "successful" than--those that set higher goals.40
Vinovskis concluded that after spending $811 million of taxpayer money between 1966 and 1991, the labs had little to show for it by way of success.
FAILED EFFORTS AT REFORM
The inefficiencies and contradictions of the regional labs have not escaped notice. In fact, part of a congressional reorganization of OERI in 1994 specifically addressed these problems.41 The reorganization included the creation of an Office of Assistance and Dissemination (ORAD), designed primarily to guide and monitor the activities of the regional educational laboratories. The labs were required to uphold the agreements in their five-year contracts with the government and to promote the goals identified by their individual boards. They also were given two quantifiable tasks: (1) bring together scattered successful reform efforts at the state and local levels, and (2) place more emphasis on existing successful reforms to promote their implementation in other areas of the country.
Thus, the 1994 reforms provided a basis for OERI action both to harness the activities of the labs and to promote greater accountability in and better guidance for their operations.42 Even Preston Kronkosky, Executive Director of the Southwest Educational Laboratory, observed that the new arrangement provided a valuable opportunity for the labs to streamline their goals and conform them to the legislation.
This
initial optimism was quickly dashed, however, by the lobbying
efforts of the Council on Educational Development and Research
(CEDaR). Established initially to represent the interests of these
labs, CEDaR apparently found the Republican-led 104th Congress as
receptive as the Democratic-controlled one before it. A major blow
to the 1994 reform came in 1996 when Congress, in its
appropriations report, claimed that all of the labs' work should be
based on the directives of their regional governing boards. This
directly opposed the stipulation tying them to their contracts with
OERI that had been imposed only two years
earlier.43
WHAT POLICYMAKERS SHOULD DO
The best way to reform the regional education laboratories is to incorporate competition and sound research standards into the process.
Labs should compete with other agencies and with the private sector for federal grants to fund each project. Moreover, given today's widespread access to the Internet and the existence of various organizations that accumulate credible information, the education bureaucracy no longer can justify wasting taxpayer dollars by charging regional laboratories with gathering information.
The labs' research methods and the activities they use to disseminate their conclusions must be overhauled. Policymakers should adopt a set of guidelines to govern decisions on how labs conduct and disseminate research. Such guidelines should include:
-
Rescuing "research" from nihilism. One problem with much of the labs' research lies in "results" that cannot be applied to other cases because the subjects studied were not controlled for all variables. Jay P. Greene, an assistant professor of government at the University of Texas at Austin and a research associate at Harvard University's Program on Education Policy and Governance, has suggested one way to improve the quality of research. He maintains that educational researchers should use "random assignment" in selecting subjects for a study, much as the medical research community does. In controlled studies using random assignment, two groups are tracked that, on average, are exactly alike except for the type of "treatment" being tested. "Thus," says Greene, "any differences observed between the randomly assigned groups after a period of time can be attributed to the treatment."44 SEDL's Hoover contends that "when you work with schools, you can't afford to conduct random experiments."45 But the alternative is conducting an experiment on students without a viable mechanism with which to measure the experiment's success.
-
Raising research standards. As Bonnie Grossen illustrates in the Fall 1996 American Educator, many experimental practices today jumped from theory or hypothesis to the classroom without first being tested by formal experimentation or analysis to determine the validity of the underlying hypothesis.46 According to Arthur Ellis and Jeffrey Fouts, professors of education at Seattle Pacific University, there are three different levels of research.47 Level I is theory-building, in which the researcher develops a hypothesis. Level II tests the theory through formal experimentation and analysis of data to determine the validity of the hypothesis. Level III replicates the results in large-scale studies and school- or district-wide implementation. Level III research calls for peer review and replication of the experiment, as well as large-scale or long-term follow-up studies. Unfortunately, many of the programs being touted by the regional labs have not had any Level III or even Level II research.48 Robert Stonehill of OERI believes that the current role of the labs is to move research from Level I to Levels II and III.49 Though this is reassuring, few parents will find it comforting that education methods based only on Level I research are being used with their children.
-
Defining what "research based" and other terms mean. At the heart of the problem are the vague definitions and misuse of federal education "research" terms and the phrase "research based." Robert Sweet, former head of the National Institute of Education (NIE), OERI's predecessor agency, has been an outspoken advocate of redefining research terms. For example, he believes such terms as "exemplary" and "promising" should be replaced with stricter terms such as "reliable and replicable" and "pilot tested"--assuming that they actually apply. The term "reliable replicable research" should mean only objective and valid scientific studies that (1) include rigorously defined samples of subjects that are sufficiently large and representative to support the general conclusions; (2) rely on measurements that meet established standards of reliability and validity; (3) test competing theories where multiple theories exist; (4) are subjected to peer review before their results are published; and (5) discover effective strategies for improving academic skills. The ultimate test of a program's usefulness should be whether the students in the program excel academically. The federally funded labs should replicate only those methods and education plans that have been proven to boost academic achievement.
CONCLUSION
Members of Congress can take a grand step to promote sound education that achieves success in America's classrooms. The solution lies in the nation's commitment to proven models of education success.
Federal education research labs, as currently funded and operated, have done little to promote these successful models. Their funding should be tied to programs that boost academic outcomes. The regional labs also should compete with other agencies and the private sector for federal grants to fund each individual project, or else their functions should be rolled over to the states to enable each state to set up its own research entities.
Finally, policymakers should establish simple guidelines to govern decisions on how to conduct and disseminate research. The sole determining factor in deciding whether a program will be funded and implemented in the schools should be an observed increase in academic scores, not how much the educational bureaucracy or education lobbyists like the program.
Nina H. Shokraii is a former Education Policy Analyst at The Heritage Foundation.
Endnotes
1. The author would like to thank Jake Phillips, a 1998 summer intern at The Heritage Foundation from Duke University, for his contributions to this study.
2. Diane Ravitch and Chester Finn, Education Reform, 1995-1996 (Indianapolis, Ind.: Hudson Institute, 1996).
3. Nancy Kober, From Promise to Practice: Stories from the Regional Educational Laboratories, U.S. Department of Education, Office of Educational Research and Improvement, September 1996; see http://www.ed.gov/prog_info/Labs/Promise/overview.html .
5. Interview with Robert Stonehill, Director of the State and Local Services Division of the Office of Reform Assistance and Dissemination, U.S. Department of Education, Office of Educational Research and Improvement, May 20, 1998.
6. This section draws on a research document prepared by Josh Smith, an intern for the House Committee on Education and the Workforce, "Overview/Analysis of ED Regional Lab Program and Other OERI Research Programs," July 7, 1997.
7. See http://www.ael.org .
11. See http://www.vlab.brown.edu .
17. U.S. Department of Education, Office of the Inspector General, "WestEd's Administration of the Regional Educational Laboratory Contracts," March 31, 1998.
19. U.S. Department of Education, Office of the Inspector General, "Review of Costs Incurred by the University of North Carolina, Greensboro, and the Southeastern Regional Vision for Education Under the U.S. Department of Education's Regional Educational Laboratories Contract," February 25, 1998.
21. U.S. Department of Education, Office of the Under Secretary, "Reports on Reform from the Field: District and State Survey Results: Final Report," prepared by the Urban Institute, Washington, D.C., 1997.
23. Luna Levinson and Robert Stonehill, "Tried and True: Tested Ideas for Teaching and Learning from the Regional Educational Laboratories," U.S. Department of Education, Office of Educational Research and Improvement, September 1997.
24. Robert M. Stonehill and Lynn M. Spencer, Profiles of the Regional Educational Laboratories, U.S. Department of Education, Office of Educational Research and Improvement, December 1997.
25. See http://www.ael.org/quilt.htm .
26. Josh Smith, "Education Research--Models, Issues, Problems," report produced for the Committee on Education and the Workforce, U.S. House of Representatives, July 31, 1997.
30. Bonnie Grossen, "Making Research Serve the Profession," American Educator, Fall 1996. See also Tyce Palmaffy, "No Excuses," Policy Review, January-February 1998.
31. Maris A. Vinovskis, "Analysis of the Quality of Research and Development at the OERI Research and Development Centers and at the OERI Regional Educational Laboratories," Final Report, Office of the Assistant Secretary, U.S. Department of Education, Office of Educational Research and Improvement, June 1993. Vinovskis served as Research Advisor to the Assistant Secretary of OERI under Diane Ravitch (Bush Administration) and Sharon Robinson (Clinton Administration).
41. "Regional Educational Laboratories for Research, Development, Dissemination, Technical Assistance, RFP No. 95-040: Statement of Work," U.S. Department of Education, Office of Educational Research and Improvement, 1995, p. 3.
42. Maris A. Vinovskis, "Changing Federal Strategies for Supporting Educational Research, Development, and Statistics," preliminary draft prepared for OERI's National Education Research Policy and Priorities Board, October 1997.
46. For example, Jean Piaget's work in developmental psychology is at Level I (basic). Piaget never tried to teach children; he only observed what children do at different stages. Project Follow-Through is an example of Level III research (the most advanced). From 1967 until the summer of 1995, this $1 billion project studied more than 70,000 students in more than 180 schools. The project's goal was to find a specific teaching method that raised the academic performance of the poor. An independent agency evaluated the study. The model found "Direct Instruction," a method that relies more on memorization and drill, the most successful at boosting academic achievement in poor children. Yet the National Institute of Education (NIE), the predecessor of OERI, discredited Project Follow-Through in 1981, saying that "the audience for Follow-Through evaluations is an audience of teachers that doesn't need statistical finding of experiments to decide how best to teach children. They decide such matters on the basis of complicated public and private understandings, beliefs, motives and wishes."
