Over the past 20 years, computers and the sharing
of information that they facilitate have penetrated nearly every
aspect of American life. Indeed, reliance on computers grows every
day, from shopping at grocery stores and filing taxes to driving an
automobile and communicating with relatives and business
associates.
This
explosion in the technology has increased efforts to equip every
classroom with computers and "wire" every school to the Internet.
Between September 1984 and September 1997 alone, the number of
computers in America's K-12 schools increased elevenfold to more
than 8 million units.1 Educators have been forced to
keep up, and some are finding themselves teaching general skills in
how to use a computer while they use them to teach other
subjects.
Few
Americans would question the role that computers could play in
education. For the United States to maintain its high-technology
status in the global economy, it seems fair to expect computers to
be given a more integral role. Some educators claim that ready
access to computers and increased use of computers in K-12
education has a beneficial effect on educational outcomes. In the
same way that computer technology has improved the operation of
automobiles, these proponents believe computers will make the
classroom a better environment in which to teach the difficult
concepts that lead to higher academic achievement. To these
educators, a computer in the classroom may become the deus ex
machina of Education in the 21st century.
But
are classroom computers delivering on this expectation? Does access
to a computer or use of a computer in instructing students improve
their academic achievement? Answering these questions is especially
critical today because politicians are proposing to spend billions
of tax dollars on expanding access to computers in schools in order
to bridge a so-called digital divide. For example:
-
President Bill Clinton has proposed a $2
billion program to increase access to computers and the Internet in
low-income neighborhoods and schools.2
-
Senator Joseph Biden (D-DE) has proposed
spending tens of millions of dollars for computer-based
instruction.3
-
Vice President Al Gore has made access to
computers in the classroom a major policy issue of the 2000
presidential campaign, calling for "[e]very classroom and library
[to be] wired to the Information Superhighway."4
- The President's Panel on Educational
technology has argued that the federal government should spend
between $6 billion and $28 billion each year on an ambitious
program of computer infrastructure development (both hardware and
software), teacher training, and research.5
Such
spending would supplement the $1.25 billion in federal money
already spent between fiscal year (FY) 1997 and FY 2000 on the
technology Literacy Challenge Fund,6 which provides
funding for new computers, software, and teacher training.
Although politicians may be quick to call
for government subsidies to increase the number of computers in the
classroom, previous research on the effectiveness of computers in
improving academic achievement has been inconclusive at
best.7 In other words, it is not clear that spending
more tax dollars on computers will boost test scores.
To
help fill this gap in the research, the author used data from the
National Assessment of Educational Progress (NAEP) to determine
whether the use of computers in the classroom has direct and
positive effects on academic achievement. The analysis showed
that:
- Students who use computers in the
classroom at least once each week do not perform better on the NAEP
reading test than do those who use computers less than once a
week.
An
important consideration in an analysis of this issue is teacher
training and preparation in the use of computers, since the
students of teachers who are not adequately trained to use them in
reading instruction may not perform as well on the NAEP reading
test as students whose teachers are adequately trained. This report
specifically analyzes computer usage in the classrooms of teachers
who responded that they are at least moderately well-prepared in
the use of computers in reading instruction.
Background
The
existing research on how academic achievement is affected by
computers in the classroom offers varying conclusions. Some
research indicates that computers may aid in achievement. Other
research concludes that computers are of questionable
effectiveness.
In
1997, Harold Wenglinsky of the Educational Testing Service, which
works closely with the National Center for Education Statistics in
preparing the NAEP data file, published a major study on computers
and academic achievement. Using data from the 1996 National
Assessment of Educational Progress math examination, Wenglinsky
analyzed student computer use both in class and at home,8 as well
as a variety of social and behavioral factors that could explain
math achievement. That study generally showed a positive reaction
to the technology. Wenglinsky noted, however, that students who
used computers predominantly for drill and practice, as opposed to
using them in ways that develop higher-order thinking skills,
tended to do worse on the NAEP math test.
The
results of other studies extolling the benefits of computer-aided
instruction are questionable because they overlook the factor of
the instructor's capabilities. Many early studies of computers in
elementary educational settings employed highly trained educational
researchers rather than ordinary teachers. Their advanced training
and experience may have facilitated the learning process, making
the effect of the computers alone difficult to
ascertain.9 Those studies suggest that students who use
computers in the classroom show at least a modest level of
achievement gain over students who do not use computers. Clearly,
the extent of teachers' computer training and their level of
preparation in using computers in Education will vary and thus
affect the level of success of computer-aided instruction.
In
recent years, criticism of previous studies on the beneficial
effects of computers and the role of computers in the classroom has
grown. Todd Oppenheimer, an associate editor at Newsweek
Interactive, has noted that each time a new technology has been
developed in the United States, whether it was Thomas Edison's
motion picture machine, the portable radio receiver, or some other
technological marvel, enthusiasts purported that these inventions
would replace and revolutionize Education in America.10
These claims have never been fully realized, and Oppenheimer is not
alone in his criticism.11 Some critics consider
computers in the classroom a mere fad, while others assert that
because computers are growing in their importance to every aspect
of society, it is better to expose children early to this evolving
technology.12 Otherwise, American students may continue
to perform more poorly on standardized tests than do their peers in
other countries.13 Clearly, the debate on computers in
the classroom is far from settled.
|
How to Interpret
These Findings
This report contains the results of statistical tests that use
NAEP data to explain differences in reading test scores. These
statistical tests isolate the independent effects of a number of
factors on reading scores (such as the Education of parents) in
order to determine whether computer use at least weekly matters to
these test scores. The statistical tests (or correlations) cover
data on a wide array of school children, as defined by their race,
income, and other socioeconomic characteristics. Because the
statistical model used here includes these socioeconomic
characteristics, the reader can interpret these findings as
applicable to each of these groups of students. Thus, the findings
about computer use and reading scores apply as much to upper-income
as to lower-income students, to blacks as to whites, to girls as to
boys, and so forth.
These correlations suggest that there is a statistical
relationship between the factor and achievement in reading, but
they do not suggest that these independent factors cause
differences in academic achievement.
The variables in the model came from the NAEP database and do
not include everything that might have an effect on academic
achievement, such as the methods used to teach reading. These
factors may be much more important in general, or for a particular
child, than the factors recorded in the NAEP data. Moreover:
-
Some variables, such as
participation in the federal free and reduced-price lunch program,
are proxies (substitutes) for other unobserved factors. For
example, eligibility for the free and reduced-price lunch program
is determined by income; only children from low-income families may
participate. Although not all low-income children will participate
in the free and reduced-price lunch program, many will. Such
information may be used, then, to analyze the effect of different
characteristics on achievement.
-
Some variables also may be used to
determine the effect of some unobservable "third factor." For
example, this model does not suggest that poor families have
children who do worse on the NAEP because they are poor.
Rather, poor families may have some unobservable characteristics or
challenges that make it more difficult to succeed in school.
Similarly, the categories of black and Hispanic students cover
children whose characteristics other than their race may make it
more difficult for them to score well.
- "Statistically insignificant" means that the effect of the
variable/factor is no different from zero effect. For example, if
the relationship between computer use at least weekly and academic
achievement is statistically insignificant, that means that those
students who use computers at least weekly do no better than those
who do not.
|
Characteristics
of the NAEP Data
The
author used the 1998 NAEP database on reading to analyze the
influence of computers on academic achievement. The National
Assessment of Educational Progress, first administered in 1969, is
an examination that measures academic achievement in a variety of
fields, such as reading, writing, mathematics, science, geography,
civics, and the arts. Currently, the NAEP is administered to 4th,
8th, and 12th grade students, and the tests for math and reading
are given alternatively every two years. In 1998, for example, the
NAEP reading test was administered; math was assessed in 1996 and
2000.
The
NAEP actually involves two tests: a nationally administered test
and the state-administered tests. Over 40 states participate in the
separate state samples that are used to gauge achievement within
individual jurisdictions. For the purposes of this study, only the
1998 national data were used.
The
most significant benefit of using the NAEP data is that in addition
to test scores in the subject area, it included an assortment of
background information for the students taking the exam, their main
subject-area teacher, and their school administrator. Responses
from the teachers and school administrators are linked to the
student's information, which yields a rich database of information.
The background questions include:
-
TV viewing habits,
-
Computer usage at home and school,
-
Teacher tenure and certification,
-
Socioeconomic status,
-
Basic demographics, and
- School characteristics.
By
incorporating this information with their assessments of NAEP data,
researchers can better understand the factors that can explain the
differences in results found among children who take the NAEP
tests.
The Heritage
Analysis
This
analysis considered the effect of computers in the classroom on
academic achievement by analyzing six factors: frequent in-class
computer use by trained teachers, race and ethnicity, parents'
educational attainment, number of reading materials in the home,
free or reduced-price lunch participation, and gender. The effect
of each factor can be isolated using regression analysis. The
Heritage model employs a jackknifed ordinary least squares
model14 and examines the effects of each factor on the
NAEP 1998 reading test's nationwide sample of public school
children.15
Independent
Variables
- Frequent In-Class Computer Use by
Trained Teachers
The effect of computers in the classroom on achievement can be
adequately assessed only when two conditions are met. First,
computers must be available and accessible for use by both teachers
and students. Second, the teacher using the computer for
instructional purposes must be versed in the operation of the
hardware and subject-matter software. The quality of
computer-assisted instruction cannot be determined simply from the
number of computers available. If teachers are not prepared to use
computer hardware and software specific to the academic subject
matter (in this case, reading), then even if there are computers
present, their students may actually learn less because of
unqualified instruction. Sherry Turkle, a professor of the
sociology of science at the Massachusetts Institute of technology,
notes that the possibilities of using a computer poorly "so
outweigh the chance of using it well, [that] it makes people like
us, who are fundamentally optimistic about computers, very
reticent."16 It is critical, then, that any model that
purports to analyze computers in the classroom and student
achievement include a variable to control for teacher
preparation.
The interaction of computer availability
and teacher preparation is critical to understanding the
effectiveness of computers in the classroom. If the analytical
model did not control for regularity of use, the relative
effectiveness attributable to the computers would be questionable.
It is impossible to assess accurately the effectiveness of any
teaching tool if the tool is not used often enough to have some
pedagogical effect. Further, if teachers are not qualified to teach
with computers, the effect of the availability of computers alone
might generate biased achievement statistics that would be limited
in their usefulness. Thus, the Heritage model considers both of
these factors to estimate the true effect of computer-aided
instruction on academic achievement.
-
Race and Ethnicity
Many studies and reports have demonstrated that over time,
African-American and Hispanic students tend to perform more poorly
on standardized tests than do white students (although the gap has
generally narrowed over the past 25 years).17 There are
a number of possible explanations for this trend.18
Because strong differences in academic achievement exist among the
races, the variables of race and ethnicity are included in the
analysis.
-
Parents' Education
Many researchers have noted that the educational attainment of
a child's parents is a good predictor of their child's academic
achievement. Parents who, for instance, are college educated could
be better equipped to help their children with homework and
understanding concepts than are those who have less than a high
school Education, other things being equal. Because the Education
level of one parent is often highly correlated with that of the
other, only a single variable is included in the analysis.
-
Number of Reading Materials in the
Home
The presence of books, magazines, encyclopedias, and newspapers
generally indicates a dedication to learning in the household.
Researchers have determined that these reading materials are
important aspects of the home environment.19 The
analysis thus includes a variable controlling for the number of
these four types of reading materials found at home.
-
Free/Reduced-Price Lunch
Participation
Income is often a key predictor of academic achievement because
low-income families seldom have the financial resources to purchase
extra study materials or tutorial classes to help their children
perform better in school. Although the NAEP does not collect data
on household income, it does collect data on participation in the
federal free and reduced-price lunch program that are used
here.20
-
Gender
Empirical research has suggested that girls tend to perform
better on reading and writing subjects while boys perform better in
the more analytical subjects of math and science.21 Many
authors have expounded on this idea,22 yet the data on
the male-female achievement gaps can lead researchers to often
inconsistent observations. For example, in 1998, young men scored
higher than young women on both the verbal and quantitative
sections of the Scholastic Achievement Test (SAT). Some writers
noted that this may be because of a fundamental bias against
females in America's educational system.23 Another
explanation, however, is that the test results reflect a selection
bias in which more "at-risk" females opt to take the SAT relative
to males.24 In order to account for this difference, the
analysis includes a variable for gender.
- Omitted variables
Previous research25 has included more family
background variables in the model specification. In the 1998 NAEP
database, the only information available on children's parents is
their educational attainment. The NAEP does not ask whether the
child lives with both parents (or parental figures), one parent, or
no parents (i.e., in a group home). Future administrations of the
NAEP test should include this type of question since a great deal
of research has found that having both parents in the home can
improve a child's academic achievement.
Results of the
Analysis
The
six factors were entered into a statistical model26 that
was then applied to the NAEP's 1998 nationwide sample of public
school children who took the reading test.27 Chart 1 and
Chart 2
show the percent change in 4th and 8th grade reading scores
attributable to the factors in the model, compared with a base
case.28 Here, the base case is defined as a child with
the following characteristics:
-
White;
-
Female;
-
Non-poor (that is, not participating in
the free and reduced-price lunch program);
-
Parents who did not attend college;
-
Has two out of the four possible reading
materials in the home; and
- Did not have weekly computer instruction
by a teacher who is at least moderately well-prepared in using
computers for reading education.
A white female child who is not poor, whose parents did not attend
college, who has two out of the four possible reading materials in
the home, and who does not have weekly computer instruction by a
prepared teacher would score 233.3 points on the 1998 NAEP (out of
a maximum of 500) in the 4th grade or 258.6 points in the 8th
grade. If she were poor, black, or Hispanic, her score would drop,
on average; if her home had more than two reading materials, or if
her parents had taken any college-level courses, her score would
increase.
For
both 4th and 8th grades, the variable for computer instruction and
teacher preparation is not statistically significant, meaning that
the effect of the variable is not statistically different from
zero. These results mean that the variable for computer instruction
shows no effect on the academic achievement of the students.
Thus, the Heritage model predicts that
students with at least weekly computer instruction by well-prepared
teachers do not perform any better on the NAEP reading test than do
students who have less or no computer instruction.29
These findings are consistent for both 4th and 8th graders. In
fact, if the variable were significant, it would indicate that
those students who were frequently taught using computers would do
slightly worse on the NAEP than those who were not. Both Chart 1
and Chart 2 show that there is a negative percent change in the
NAEP reading score for the computer variable. Such a result might
indicate that children are not learning critical higher-order
thinking skills that achievement exams like the NAEP aim to test.
Further, these results are consistent with Wenglinsky's analysis of
1996 NAEP math data.30
At
the same time, variables such as race, income, home environment,
and parents' college attendance are all significant factors in
explaining differences in reading test scores.
Both
4th and 8th grade girls score slightly higher than do boys on the
NAEP reading exam, a fact that bolsters recent evidence on gender
differences in academic achievement. American Enterprise Institute
W. H. Brady Fellow Christina Hoff Sommers notes that girls on
average "get better grades, are more engaged academically, and are
now the majority sex in higher education."31 The results
here support the contention that schools are not shortchanging
girls.32
Conclusion
As
this analysis shows, the use of computers in the classroom may not
play a significant role in explaining reading ability. Thus,
dedicating large amounts of federal tax dollars to the purchase of
computer hardware, software, and teacher training could crowd out
other worthwhile Education expenditures on, for example, new
textbooks, music programs, vocational Education, and the arts. This
report does not suggest that there is no place for computers in the
classroom. It does, however, demonstrate that computers may not
have the effect on academic achievement in reading that some might
expect, even when they are used by well-trained instructors.
Kirk A. Johnson,
Ph.D. is a Policy Analyst in the Center for Data Analysis at
The Heritage Foundation.
Appendix A: Results of the Statistical
Models
Table 1 reports the results of the
Heritage analysis of data from the National Assessment of
Educational Progress (NAEP) on reading in the 4th and 8th grades.
As shown in this table, the variables in the Heritage model are
statistically significant,33 with the exception of the
socio-economic factors-other non-white communities variable in the
8th grade analysis and the computer variable analyzed in this
report.34
In analyzing the effects of computers in the
classroom, there are two statistical issues to consider. First, the
NAEP exam is a long test and therefore is not administered in its
entirety to all children. Rather, different parts are given to
different children. Certain students will do better on certain
portions of the test than others. Consequently, a "true" score must
be estimated, or imputed, from the incomplete information. The NAEP
estimates five plausible composite reading scores and recommends
that researchers use all five in any analysis. The Heritage model
used in this analysis follows the guidelines specified by the
Educational Testing Service (which works closely with the National
Center for Education Statistics in developing the file) for
incorporating all five reading scores into the
analysis.35
Second, the NAEP utilizes a complex sample
design that oversamples children with certain
characteristics.36 Each child is assigned a unique
weight calculated from the probability of being selected from the
population at large (in this case, from the U.S. population of 4th
or 8th graders in public schools). The NAEP's sample design
requires a complex modeling technique, which the Heritage model
employs.37
U.S. Bureau of the Census, Statistical
Abstract of the United States, 1998 (Washington, D.C.: U.S.
Government Printing Office, 1998), Table No. 281, p. 179.