While presenting his budget before a joint session of Congress on February 27, President George W. Bush declared that "[m]easuring is the only way to know whether all our children are learning." Echoing the principles articulated in his education plan, No Child Left Behind, the President went on to outline his plan to infuse accountability into federal education spending. For the first time, states would be required to demonstrate annual increases in academic achievement, especially among disadvantaged students and English-as-second-language learners.
President Bush sees testing children as the way to verify that states are truly improving achievement. Under his plan, state-developed tests would be used to measure the success of states and their schools.
In order to confirm state progress on state assessments, the plan calls for an annual sampling of 4th and 8th grade students on the National Assessment of Educational Progress (NAEP) in math and reading. But what exactly is the NAEP test? How is it administered? Who runs it? And how would it have to be retooled to fit the President's plan?
WHAT IS THE NAEP?
Commonly known as the "The Nation's Report Card," the NAEP was first administered in 1969. The examination measures academic achievement of 4th, 8th, and 12th grade students. Assessments developed by a 26-member National Assessment Governing Board (NAGB) are used to test reading, writing, mathematics, science, geography, civics, the arts, and other fields. Tests in math and reading are given more often than those for other subjects.
Unlike other tests, the NAEP does not provide information about a particular school's or student's performance. Rather, it is designed to provide a general picture of the levels of skill and knowledge among students nationwide or in a particular state. Only a small sample of students statewide are tested, and no student takes the entire test. The scores of individual students and schools are not released.
In other words, the NAEP can reveal whether students in a particular state are reading at a proficient level as determined by a NAEP standard. It can show how states rank when these statistics are compared. But it cannot show whether a particular student is reading proficiently or how his or her school compares to other schools.
The NAEP provides a rich database on educational performance and student background. From the test, data on such factors as teacher qualifications, socioeconomic status, computer usage, hours spent watching television, reading habits, and other demographic and school information can be gleaned. Such information is valuable to education reform groups because researchers are able to isolate factors, such as the number of reading materials in the home, that correlate to higher achievement.
The NAEP is a criterion-referenced test. This means it is designed to show how well students know the body of information and skills according to specified criteria. Scores are arranged in three categories: basic, proficient, and advanced.
Other well-known tests, such as the Iowa Test of Basic Skills (ITBS), Stanford 9 (SAT-9), and Scholastic Aptitude Test (SAT), are norm-referenced, meaning they measure how well a student knows the content compared with a representative sample of students. The results of these tests thus are given as a percentile rank: A student in the 90th percentile, for example, has scored higher than 90 percent of his peers.
HOW IS THE TEST ADMINISTERED?
Three types of NAEP tests are used to measure academic achievement at the national and state levels. They are administered using separate examinations, samples of students, and data collection procedures.
-
The "Main" NAEP and "Long-term" NAEP are administered to national samples of students. Generally, subjects are not tested more often than every four years. The Main national test measures academic achievement nationwide based on current trends in curricula and education practices according to the National Assessment Governing Board. The Long-term NAEP does not change from year to year because its purpose is to show trends spanning the 30 years of the program; thus, questions on this test are the same every year it is administered.
-
State NAEP tests are given to samples of students within participating states. The sample size is 2,500 students per subject per grade. Although the test is paid for with federal funds, states pick up the extra cost to train teachers and bring in additional personnel. State NAEP tests allow states to compare themselves with other states using a uniform test developed by the NAGB. In 2000, 41 states participated in the state NAEP testing program.
NAEP state testing was introduced in part because state assessments revealed inconsistent data, making comparisons difficult. Over the years, the NAEP has influenced in-state tests, which have moved toward greater alignment with the NAEP in content, rigor, and testing methodology. In addition, poor NAEP results have triggered teaching changes in some states. In 1994, for instance, California ranked last in 4th grade reading proficiency. The legislature responded by enacting the California Reading Initiative, which replaced discredited whole language reading instruction with research-based teaching methods.
WHO RUNS THE NAEP?
The NAEP is administered by the National Assessment Governing
Board. Created by Congress in 1988, the board is comprised of a
bipartisan group of governors, state legislators, local and state
school officials, educators, business representatives, and members
of the public. The 26 members are appointed by the Secretary of the
U.S. Department of Education after nomination by the board. The
term of service is three years, and members may not serve more than
two terms.
According to the authorizing statute, the Improving America's Schools Act of 1994 (Public Law 103-382), the duty of the board is "to develop assessment objectives and test specifications through a national consensus approach which includes the active participation of teachers, curriculum specialists, local school administrators, parents, and concerned members of the public." Groups that represent the categories specified in the law (for example, the Republican Governors Association and the National Education Association) make recommendations to the board, as do Members of Congress, education policy organizations, and others.
In addition, to ensure diversity of membership, the statute states that "the Secretary and the Board shall ensure at all times that the membership of the Board reflects regional, racial, gender, and cultural balance and diversity and that the Board exercises its independent judgment, free from inappropriate influences and special interests."
The authorizing statute includes several provisions designed to ensure that the NAEP is not used as a federal test or to undermine student privacy. The public has access to data, questions, and test instruments. It is illegal to disclose assessment data that would identify individuals or individual schools.
The standards that describe what students should know and be able to do are designed to constitute a nationally accepted base of necessary knowledge and skill. The framework and the assessments are developed through a consensus approach involving teachers, curriculum experts, policymakers, business representatives, and members of the public.
HOW WOULD THE PRESIDENT'S PLAN CHANGE THE NAEP?President Bush would use the NAEP to confirm the progress shown in state tests. In order to perform the task envisioned in his education plan, there would have to be changes in the way the NAEP is administered. For example:
-
The Bush plan calls for the NAEP to be administered every year and for the results to be reported annually. Currently, it takes around 18 months for results to be released. The cost of expansion could raise the NAEP's overall cost from $40 million to $110 million per year.
-
Because the NAEP uses sampling techniques, much as an opinion poll does, there is a margin of error of two or three scale points. On an annual test, small achievement gains could be within the margin of error and therefore not a reliable short-term indicator of progress by a state. This problem would have to be addressed.
-
Concerns about the NAEP's indirect influence on the content of state tests also would have to be addressed. Critics of the NAEP and other national tests worry that linking test results to federal funds would push states toward a national curriculum.
-
The NAEP is not completely independent of the Department of Education, and this raises the risk of politicization. While the National Assessment Governing Board sets policy, NAEP operations remain within the department. For instance, the selection of contractors to write and administer the test is the responsibility of the Commissioner of Education Statistics. To preserve the NAEP as an independent test, the NAGB would need to be given authority over operations.
CONCLUSION
Improving achievement has been the unrealized goal of federal
education programs for decades. Under President Bush's education
plan, No Child Left Behind, federal funding would be linked
to whether states actually succeed in this endeavor, as measured by
the NAEP test. Ronald Reagan once said, "Trust but verify," and the
Bush Administration is seeking to use the test to confirm state
achievement trends.
The NAEP currently provides a dynamic source of state and national achievement data. Consistency in sampling, privacy protections, security against fraud, and comparability make it a useful testing instrument. However, the test has its limitations, and there are concerns about its possible impact on teaching and local control. To restructure the NAEP to fit its proposed new role, lawmakers must address the limitations and concerns.
Krista Kafer is an Education Policy Analyst at The Heritage Foundation.
