Sam Dillon, "U.S. Students Remain Poor at History, Tests Show", NYT 6/14/2011
American students are less proficient in their nation’s history than in any other subject, according to results of a nationwide test released on Tuesday, with most fourth graders unable to say why Abraham Lincoln was an important figure and few high school seniors able to identify China as the North Korean ally that fought American troops during the Korean War.
Over all, 20 percent of fourth graders, 17 percent of eighth graders and 12 percent of high school seniors demonstrated proficiency on the exam, the National Assessment of Educational Progress. Federal officials said they were encouraged by a slight increase in eighth-grade scores since the last history test, in 2006. But even those gains offered little to celebrate because, for example, fewer than a third of eighth graders could answer even a “seemingly easy question” asking them to identify an important advantage American forces had over the British during the Revolution, the government’s statement on the results said.
Sam Wineberg, "Can even educators answer these lame questions?", HNN 6/19/2011:
“Ignorance of U.S. History Shown” was the New York Times headline. Less than a quarter of American students were able to name two contributions to U.S. history by Abraham Lincoln and Thomas Jefferson.
A report about last week’s NAEP findings? Try again.
What about this? “Test Shows Knowledge of American History Limited.”
Neither headline was written about the latest NAEP. For the first, we’d have to dust off a test given to college students by the New York Times in 1943. For the second, we’d have to reach back to 1976, our bicentennial year, when the Times ran another survey with Bernard Bailyn's help. Bailyn threw up his hands in disbelief that more students thought the Puritans “guaranteed religious freedom” than not. “I don’t know how to explain it,” he said.
With all due respect to the acclaimed Harvard historian, I do.
Indeed, my powers exceed explanation. I can predict the future. In March 2004, seven years before the release of the last week’s NAEP findings, I wrote the following in the pages of the Journal of American History: “When the next national assessment rolls around in 2010 do not hold your breath for the headline announcing ‘U.S. Children Score Well on the 100 Most Basic Facts of American History.’ The architecture of modern psychometrics ensures that that will never happen—no matter how good a job we do in the classroom.”
Let me come clean. Here's two tips for predicting the future about students’ historical knowledge.
First, know a bit about modern test construction. No matter what the NAEP spokespeople tell us about the NAEP being “criterion referenced”—or tied to an absolute standard rather than fitted to a bell curve—multiple choice items work by tricking test-takers with “distracters.” An item is “bad” if almost everyone gets it right. So, if during the piloting testing of NAEP, it is determined that most twelfth-graders can identify “The Star-Spangled Banner,” Rosa Parks, the dropping of the bomb on Hiroshima, slavery as a main cause of the Civil War, the purpose of Auschwitz, and Harriet Tubman, these items are all thrown out because they fail to “discriminate” among students.
And I myself have the psychic power of predicting how the results will be spun — by the media, by the education establishment, by the domain experts, and by the politicians. The media's take is predicted by the bias towards "sensationalism, conflict, and laziness", as Jon Stewart put it in a recent interview with Chris Wallace. The education professionals will see things in terms of the urgent need for more of their guidance in reforming the educational system. The domain experts will point out that their vital field needs more emphasis and more funding.
As for the politicians, they'll blame the sad effects of others' mistaken or even evil agendas, and the failure to give enough power to their own faction. A lovely recent example by Rick Santorum:
"America's a very young country. Our memory is very short. And as a result, we- we don't know our- my goodness, we don’t even know our own history. There was a report that just came out last week that the worst subject of children in- in American schools is — not math and science — it's history. It’s the worst subject. How can we be a free people, how can we be a people that fight for America, if we don’t know who America is or what we’re all about? This is a-, ((in)) my opinion, a conscious effort on the part of the left, who has a huge influence on our curriculum, to desensitize America to what American values are, so they're more pliable to the new values that they would like to impose on America."
Of course, any of these self-interested reactions might happen to be correct in a particular case, since a stopped clock sometimes tells the correct time. But for a more complete analysis of the response to earlier test result, in which the stopped clocks were not very accurate, see "Freedom of speech: More famous than Bart Simpson", 3/3/2006.
And if you want to dig into the most recent NAEP U.S. History results in order to evaluate the rhetoric of the various responses, you can start here. Right up front you'll find the first unused possible headline: "Student performance improves at all grades since 1994″. Download the full report, and you'll find striking numbers and graphs like this one:
Or this one:
Of course, in order to join the (extensionally non-existent) celebration of these encouraging trends, you'd need to accept that the NAEP's psychometricians have succeeded in their goal of making the tests "criterion referenced" in a way that yields numbers that are comparable across the decades. Without looking further into the details, I don't know whether this is a plausible belief or not. My goal here is just to echo Prof. Wineberg's observation that the responses to the NAEP 2010 U.S. History test results have been spun, essentially by everyone involved, in a predictably negative direction.
[I should note that the History News Network deserves kudos for publishing not only Sam Wineberg's evaluation, but Paul Burke's "Wrong 'Correct' Answers: The Scourge of the NAEP", 6/21/2011:
The National Assessment of Educational Progress (NAEP) has three big flaws in measuring history knowledge. First, students get neither a grade nor any result from the test, so student effort is entirely voluntary. Second, NAEP seeks proficiency on so many topics it encourages shallowness, not depth. Third, NAEP marks wrong many of the correct answers. […]
In addition to giving some telling examples of question-design and scoring problems, Burke notes the serious problem that these tests share with the infamous Collegiate Learning Assessment:
For students these tests do not count towards a grade, or even a star, so while students have to show up, they have no incentive to give their best answers, or indeed to make any effort whatsoever. The tests are given in late winter, often a time of low energy and motivation in school anyway. The tests are one more intrusion in the most dismal part of the school year. It would not be surprising for students to put little energy in the answers, and even to give wrong answers as a protest.
If you're looking for something to bewail, forget about American schoolchildren's knowledge of U.S. history — consider instead the fact that in order to get a rational assessment of the NAEP U.S. History results, you need to go to some relatively obscure new-media website rather than to elite mainstream outlets such as the New York Times or the Wall Street Journal.]