Today I would like to discuss the future research potential for the NAEP, followed by some thoughts on how to improve the NAEP background questionnaire. Finally, I will cover the issue of less frequent comprehensive background questionnaires. Before I begin, let me stress that the views I express are entirely my own, and should not be construed as representing any official position of The Heritage Foundation.

Research Potential for NAEP
I would like to echo the sentiment of the framework on the unique comparative advantage NAEP possesses for research purposes related to student achievement. NAEP is the only cross-sectional micro-database on student achievement that has been collected for more than three decades. Now in the twenty-first century, its prominence has only increased with the advent of the No Child Left Behind legislation. NAEP is now the centerpiece educational achievement test whose results will be discussed and scrutinized by not only researchers and policymakers, but by teachers, administrators, other education practitioners, and the general public nationwide.

Because of the increased sample size and policy importance of the NAEP test in particular, the usefulness of the NAEP for researchers has now increased dramatically. NAEP is quickly becoming the first source for education researchers to turn for answers to educational achievement questions. Indeed, at the recent American Educational Research Association conference in Chicago, there were at least 20 sessions dealing specifically with various aspects of the NAEP, with dozens of other individual papers touching on some aspect of the survey.

Parental Questionnaires Reconsidered
Today's hearing is especially timely, given that the background questionnaire is key to making the data more usable and relevant to researchers and policy makers alike. I applaud the Board for spending so much time and effort on this worthwhile issue. Improving the non-cognitive items is the best way to help survey and inform the education debate in America today.

One issue that the Board should reconsider, though, is the potential use of parental background questionnaires. As noted in the framework, parental questionnaires have a number of positives and negatives. Parent questionnaires would yield, potentially, much more background information than is currently collected. At the same time, though, such parent surveys are both intrusive and burdensome for both the respondent and data collectors alike. The question is what, on balance, is better from the standpoint of research and policy? Is the value to researchers and policymakers sufficient to warrant a fairly intrusive and administratively challenging parent survey?

As the Board knows, a number of researchers have noted the value of quality socioeconomic status variables in analyzing student achievement, and the No Child Left Behind Act mandates that information on socioeconomic status be reported along side of test scores. At the same time others, notably Michael Podgursky, has remarked of the potential problems associated with asking children, especially younger children, about the demographics of their parents. When asked about their parents' educational attainment, for example, they often do not know or give incorrect answers.

Clearly, a parental questionnaire would help this problem a great deal, but it would have other value as well. Such a parent survey could also ask a number of other questions potentially related to their children's learning. For example, information could be gathered about the work schedule of parents and how much time parents are able to spend with their children in the evenings. If a parent works primarily in the evenings during the week, for example, then he or she may not be available to help his or her child with day-to-day homework on a regular basis.

Such a survey also has other important uses. Currently, administrative records on free and reduced price lunch eligibility are used as a proxy for poverty. If a parent questionnaire was utilized, actual income information could be asked, and researchers could assess the variability of achievement by income strata, something that cannot be done currently.

If for no other reason, NAEP should attempt a parent questionnaire because of its size and importance in the policy world. Congress and the general public will look to the NAEP first to analyze academic achievement at a national level. Therefore, NAEP should have the most comprehensive information available.

For these reasons, I suggest that the Board conduct a trial parent questionnaire and assess the strengths and weaknesses of an actual administration of a parent questionnaire along side the student test. If after the survey is taken the administrative and respondent burden is a serious roadblock to effectiveness, then it need not be attempted again. If it is seen to be effective, then it should be collected on a rotating basis as part of the comprehensive data efforts outlined in chapter three of the draft framework.

Some might argue that such a parent questionnaire is not worthwhile because of its intrusive nature. Many agencies in the federal government regularly ask questions that are far more intrusive than this one would be. For example, the Federal Reserve Board asks questions on detailed financial information, and other agencies field surveys on crime, drug use, sexuality issues, and other sensitive topics. Therefore, such a parent questionnaire would not be any more intrusive than other federal surveys, and indeed would be far less intrusive, comparatively speaking.

Finally, I would like to underscore the September 2002 comments of Dr. David Armor. Detailed socioeconomic status variables are of key importance to the NAEP, even if collected only on a student level. Family background information related to how many parents and siblings live at home, parental education, and number of reading items in the home would continue to have value for the research and policy worlds.

Possibilities for Allocating Scarce Resources
I would like to conclude today by discussing the issue of allocating scarce resources to background questionnaires. NAEP researchers such as myself will always want more data collected more often that can be used to answer more questions, but I also know that this must be balanced against scarce budget resources.

Many NAEP researchers often focus first (if not solely) on the basic skills of reading and mathematics for their analyses. Others may branch out into other subjects, limited to only the secondary fields of writing or science. Further, some state and local education policy makers are stressing a "back to basics" curriculum model focused on improving basic reading and math skills of their students.

These core subject areas are the most important ones on which NAEP should focus. Expending scarce resources on extensive student and possibly parent questionnaires for some of the other topic areas, such as history, civics, arts, and geography, may be useful to some researchers and policy makers, but this benefit should be weighed against the cost. Clearly, far fewer analysts are interested in these tertiary subjects for research; therefore, the Board might consider curtailing the background questionnaires accompanying these tests entirely. Naturally, all NAEP tests should have a basic core set of background questions, but limiting the comprehensive questionnaire to reading and math (and possibly writing and science as well) would be a way to allocate scarce resources effectively.

At the same time, using other survey instruments such as the Schools and Staffing Survey (SASS) to bolster the NAEP for school-wide data is certainly warranted, given the overlapping interests of the two surveys. As part of the SASS program a "special overlap sample" with NAEP was created in 2000. The main part of the sample minimized overlap however, as the National Center for Education Statistics wanted to spread out the burden across schools, according to Keith Rust at Westat.

An analysis of this effort should be undertaken, if it has not taken place already. So long as the overall respondent burden does not substantially decrease the response rate of either survey individually or combined, a strategy of using SASS to supplement NAEP should be adopted in the future.

Conclusion
In closing, never has the research potential of NAEP been greater, and never has the policy world focused so much attention on the survey. It is incumbent that the background information be comprehensive and of high quality. At the same time, these data efforts need to be conscious of cost and overall burden to the agencies and respondents.

Once again, I would like to thank the Board for inviting The Heritage Foundation to participate in this hearing, and I am happy to answer any questions.

         *******************

The Heritage Foundation is a public policy, research, and educational organization operating under Section 501(C)(3). It is privately supported, and receives no funds from any government at any level, nor does it perform any government or other contract work.

The Heritage Foundation is the most broadly supported think tank in the United States. During 2002, it had more than 200,000 individual, foundation, and corporate supporters representing every state in the U.S. Its 2002 contributions came from the following sources:

Individuals 61.21%

Foundations 27.49%

Corporations 6.76%

Investment Income 1.08%

Publication Sales and Other 3.47%

The top five corporate givers provided The Heritage Foundation with less than 3.5% of its 2002 income. The Heritage Foundation's books are audited annually by the national accounting firm of Deloitte & Touche. A list of major donors is available from The Heritage Foundation upon request.

Members of The Heritage Foundation staff testify as individuals discussing their own independent research. The views expressed are their own, and do not reflect an institutional position for The Heritage Foundation or its board of trustees.