Pundits were confused and inaccurate
« previous post | next post »
Also, the sky turns out to have been blue much of the time, and early returns are strongly suggesting that water is often wet. John Sides, "2012 Was the Moneyball Election", The Monkey Cage 11/7/2012:
Barack Obama’s victory tonight is also a victory for the Moneyball approach to politics. It shows us that we can use systematic data—economic data, polling data—to separate momentum from no-mentum, to dispense with the gaseous emanations of pundits’ “guts,” and ultimately to forecast the winner.
Timothy Egan, "Revenge of the Polling Nerds", NYT 11/7/2012:
In the last days of the election, Peggy Noonan had a “feel” that things were moving Mitt Romney’s way. George Will was more cerebral: his brain told him it would be Romney in a rout. And Michael Barone, who used to have a good divining rod to go along with an encyclopedic knowledge for all numbers political, also predicted a Romney landslide.
What they had in common, aside from putting up a brick Tuesday that completely missed the electoral net, was a last-hurrah push for the old-fashioned prediction by gut.
This was the year the meta-analyst shoved aside the old-school pundit. Simon Jackman of Stanford, Sam Wang at the Princeton Election Consortium and, of course, our colleague Nate Silver, all perfected math-based, non-subjective models that produced predictions that closely matched the outcomes.
Others who threw up a predictive brick, as Egan notes, are Karl Rove, Newt Gingrich, and Dick Morris.
Specifically, here's a map of Nate Silver's pre-election predictions:
And this morning's results:
In fact, Silver did better than he really should have, in the sense that 50 unfair coins, calibrated as per the probabilities that he projected for the 50 states, would be rather unlikely to come up 50 times on the side assigned the (sometimes slim) advantage. This has naturally produced the "Is Nate Silver a Witch?" meme:
Still, these results allow us to hope that the idea of learning from successes and failures in predicting objective reality will gradually have an impact on the composition and behavior of the commentariat.
This idea is a surprisingly controversial one in many fields. For example, it didn't take over the engineering disciplines known as "Human Language Technology" (speech recognition, machine translation, etc.) until roughly the decade between 1985 and 1995, as described in some slides from a presentation that I gave last summer to a National Academy of Sciences committee on Responsible Science, as revised for a presentation to a recent NSF workshop on The Future of Survey Research. And similar issues in the scientific study of language emerged during the back-and-forth last year between Noam Chomsky and Peter Novig (see the discussion and links in "Straw men and bee science", 6/4/2011).
Mark Coddington's essay "Why political journalists can’t stand Nate Silver: The limits of journalistic knowledge" — the serious perspective behind the "Is Nate Silver a Witch" joke — raises issues that will be familiar, mutatis mutandis, to observers of the human sciences.
Q. Pheevr said,
November 7, 2012 @ 8:49 am
For some people, a statistical model doesn't have to be very advanced to be indistinguishable from augury.
James said,
November 7, 2012 @ 9:22 am
Fits with the breaking news from xkcd, too.
Bummer said,
November 7, 2012 @ 12:22 pm
I just assumed that Silver had stepped out of the room when they passed out the "swing state" Kool-Aid that all the networks were swilling. I won't pretend his call of Iowa and Virginia don't fill me with otherwordly dread, but Nevada, New Mexico, and Minnesota always seemed like squeakers, yes, but nothing on the level of the real and terrifying swinginess of a Florida or Ohio. Depending on your fortitude for ingesting huge amounts of internet news, even Colorado and Wisconsin might not have looked that swingy.
Jerry Friedman said,
November 7, 2012 @ 12:39 pm
The sad bloggers mock their own presage.
[(myl)
And thou in this shalt find thy monument,
When tyrants' crests and tombs of brass are spent.
]
Nate Silver, re-aggregator « Manifest Density said,
November 7, 2012 @ 1:04 pm
[…] toward Silver through the more familiar lens of what's been happening elsewhere.UPDATE: Another perspective. I think a preference for simplistic accounts is made possible by structural factors like those […]
diogenes said,
November 7, 2012 @ 2:34 pm
what would be really great is a map where the areas of the states are proportional to their electoral college votes. The map shown above shows tht Romney won over the land-mass of the USA. A college-weighted map would show that Washington DC is the same size as Montana.
Greg Morrow said,
November 7, 2012 @ 3:19 pm
Diogenes: Sam Wang and the Princeton Election Consortium have you covered: http://election.princeton.edu/electoral-college-map/
Daniel Barkalow said,
November 7, 2012 @ 3:35 pm
According to Sam Wang, Nate Silver just assigned too large a margin of error to his predictions. There's also the possibility that Nate Silver's probabilities were based on the chance that this would be the year that voters start lying to pollsters en masse, which would mean that those coins aren't really independent. In any case, while Nate Silver's model that included things other than polls was pretty much perfect, so was Sam Wang's model that only used polls– if you just looked at the confidence intervals on the polls and combined them in the obvious way, you got the right results and the right probabilities, down to knowing that you don't know FL and do know everything else, and giving even odds of a automatic recount.
Of course, Nate Silver's models are also supposed to be more useful earlier in the process, and involve estimating the changes of events that convince undecided voters. Sam Wang's models also worked a long way out this year, but that actually just implies that nothing especially influential happened during the campaign season.
Charlie Loyd said,
November 7, 2012 @ 4:24 pm
I run the site, and it’s raised an issue that might be of wider interest on this blog: is “witch” only for women?
I contend that it isn’t. I’ve used three examples:
— Historical: Malory has some people refer to Merlin as a “wytche” in “Le Morte D’Arthur” (1485).
— Academic/specialist: anthropological usage seems to be totally genderless, for example in E. E. Pritchard’s seminal “Witchcraft, Oracles and Magic Among the Azande”.
— Contemporary colloquial: Bender uses it to refer to a man in an episode of “Futurama”.
But I recognize that, for many speakers, “witch” is at least primarily for women. I would be very interested to see professional commentary on how this word has interacted with gender over the last 500 years or so.
The other Mark P said,
November 7, 2012 @ 4:43 pm
Did any commentators predict Romney that weren't Republican/Conservative?
It's not like Newt Gringrich is every going to say that things are going swimmingly for the Democrats. The Republicans could lose every seat in the Senate and he would still be talking up their chances.
I would suggest there is little wrong with the predictive powers of the people mentioned. They just didn't like the result, and were not going to say what they actually thought in public.
Stan Carey said,
November 7, 2012 @ 4:44 pm
Both the OED and AHD5 begin their definitions of witch thus: "a person, especially a woman…"
For most of the word's history it has been associated principally with women, but the popular female witch vs. male warlock/wizard division is an obvious oversimplification. I've known males who self-identify as witches.
Jane Mills, in Womanwords: A Vocabulary of Culture and Patriarchal Vocabulary, says:
Erin Brenner said,
November 7, 2012 @ 4:54 pm
I've often thought that "witch" was for a woman and "warlock" for a man. "Wizard" is also for a man but is a higher class of magic than a witch or a warlock. I can't say where that idea came from, other than general fantasy reading for the last 30 years. Maybe "Bewitched" is to blame for "warlock."
"The Encyclopedia of Fantasy" (John Clute & John Grant, 1997) says at "witchcraft" that witchcraft is performed by women, or "witches," in modern fantasy. Some writers use "warlock" for a male witch, but more often men practice "sorcery," as "wizards" (see also the "wizards" entry). Yet "witchcraft" and "sorcery" are indistinguishable, say the authors.
The encyclopedia also notes that during the witch persecutions in the 15th-17th centuries, "witch" was applied equally to men and women. I can say, though, in Salem, MA, only one man was killed during the Witch Trials: Giles Cory. He was pressed to death for not confessing that his wife was a witch. All the others were women, accused of being witches and hanged to death.
The entries for "witchcraft," "witches," and "wizards" list major works and how they used these concepts.
diogenes said,
November 7, 2012 @ 5:08 pm
it's the wrong forum but do those electoral college weightings reflect reality?….D's say yes and R'a say no.
Keith M Ellis said,
November 7, 2012 @ 6:05 pm
@Erin Brenner, my impression is that there's been a trend in genre toward distinguishing between some of these terms on the basis of morality — warlock and sorcerer connote evil magic, while wizard does not. Sorcerer is probably the more ambiguous of the two; I think that warlock pretty strongly connotes evil.
But both witch and wizard remain quite (but not absolutely) gendered; warlock and sorcerer, as well, although they're more ambiguous as those are minority usages. What I've noticed is that genre writers (fiction or film/tv) tend to create magical worlds that feature either witchcraft or wizardry and then, as a result of that, when they want to introduce magic-using protagonists (that is, in most cases a character aligned with good) who are of the minority gender they just make the term gender-neutral. So you end up with male witches in stories about witches and female wizards in stories about wizards.
Sometimes the old gendered cultural conventions interact in all this in complicated ways — for example, in Pratchett's books he established the gender distinction between witches and wizards and then supported it as a qualitative difference in the magic itself which, naturally, reflects gender norms (witchcraft is intuitive and a hearth-magic, wizardry is abstracted and scholarly and the result of institutionalized education). But then he allows for a (one!) female wizard, which seems progressive but arguably is more a kind of tokenism. I stopped reading his books years ago, but as far as I know, he's never introduced a male witch. And why would he? It's easy for many people to imagine a woman wanting to behave like a man; much more difficult is imagining why a man would want to behave like a woman. (So, in our culture today we've more quickly come to approve of female physicians than we have of male nurses.)
At any rate, changing gender norms have caused some evolution in the use of these terms. I recall watching the US TV show Bewitched as a child and it differentiated witch and warlock purely and universally by gender. Likewise, the old comic book Wendy the Good Little Witch had exclusively female witches. But there's more interest in gender diversity in these sorts of stories today coupled with a lessened interest in the use of strict gendered terms. I think that this has opened up the territory vacated by gender to occupation by moral alignment.
maidhc said,
November 7, 2012 @ 8:46 pm
A "water witch" is usually male.
Ray Dillinger said,
November 7, 2012 @ 9:56 pm
When I learnt the terms, "Witch" described an adherent of a pantheistic nature-oriented religion and "Warlock" the status of a person who had betrayed or abandoned the values of that religion. Frequently the latter was heard as part of the phrase "Warlock Oathbreaker" and always it was treated as a term of opprobrium. Both terms were treated as completely gender-neutral.
Fantasy books, which inform more standard or majority usage, have been another matter altogether.
Ray
Jean-Michel said,
November 8, 2012 @ 1:06 am
@diogenes/Greg Morrow: The L.A. Times has another map where each state is represented by squares, with one square per elector. It only vaguely approximates the states' actual shapes (or doesn't even try, in the case of some three-square states), but it's an interesting alternative approach.
Of Polls, Political Pundits, Predictors and I | elcidharth said,
November 8, 2012 @ 4:39 am
[…] Silver batted 1.000, which is pretty surprising to him and others who understand statistics. See http://languagelog.ldc.upenn.e… "Pundits were confused and […]
Matt_M said,
November 8, 2012 @ 5:03 am
@Ray Dillinger:
In the online etymology dictionary, the development of the word warlock is described thus:
O.E. wærloga "traitor, liar, enemy," from wær "faith, a compact" (cf. O.H.G. wara "truth," O.N. varar "solemn promise, vow;" see very; cf. also Varangian) + agent noun related to leogan "to lie" (see lie (v.1)). Original primary sense seems to have been "oath-breaker;" given special application to the devil (c.1000), but also used of giants and cannibals. Meaning "one in league with the devil" is recorded from c.1300. Ending in -ck and meaning "male equivalent of witch" (1560s) are from Scottish.
Kathleen said,
November 8, 2012 @ 7:45 am
Although more women than men died in the Salem trials, several men were also executed: George Burroughs, George Jacobs, John Proctor, John Willard, and Samuel Wardwell. Another man died in jail.
As a historian, I would say "witch" sounds right for both genders at least through the 1600s. As a reader of modern fantasy novels, I agree with commenters above who say "witch" is usually female and "wizard" or "warlock" more often male.
Rodger C said,
November 8, 2012 @ 8:34 am
Iirc, in OE. a male witchy-person is wicca, a female is wicce. They merged in ME. and are now being distinguished again by new devices.
bks said,
November 8, 2012 @ 10:53 am
It should be noted that Gott and Colley got darned close (304-234) with a model so simple that a high school student could implement it with paper and pencil:
1) For each state find the median of the preceeding month's polls
2) Add the electoral votes for that state to the total for the indicated candidate.
That's it! Now you're smarter than Karl Rove, Michael Barone and George Will.
http://www.colleyrankings.com/election2012/
–bks
Adrian said,
November 9, 2012 @ 1:55 pm
Speaking as someone who's been interested in psephology for many years, there was never any doubt in my mind that Silver and his ilk were predicting the result accurately. Not only that, but the result in most states has been clear for several months, as you can see by the way Drew Linzer's predictions never changed. New Hampshire, Iowa, Virginia, Nevada and Ohio went Obamacrat a long time ago and it would take someone very charismatic to overcome that.
Most people here in Britain felt the same as me, judging by the very short odds on an Obama victory at British betting sites. The odds were longer at American sites because Americans were obviously less objective about the election and there was a lot of wishful thinking fed by the right-wing media. As others have said, stations like Fox were always going to spin the campaign in such a way as to boost their guy, whatever they really believed. (Considering that most of Murdoch's other networks around the world have a good reputation for impartiality, I wonder whether this result will finally convince him to introduce a more objective editorial policy at Fox News?)
There were, however, factors that could've muddied the waters at the last minute, like for example the weather on polling day, whether certain demographic groups would stay at home (or not stay at home) more than had been allowed for, or whether there was any interference in the election, for example from groups obstructing certain groups of voters, or malfunctioning equipment, or lawyering. There were enough unknowables to make me a little uneasy about the result despite my faith in the statisticians.
Eneri Rose said,
November 19, 2012 @ 11:54 am
I think the Electoral College map of the USA at:
http://www.colleyrankings.com/election2012/
is very clever, although the shapes of Michigan and Massachusetts visually dilute their respective population sizes.
And I must say, I cringe when I hear people say elecTORal college. We have eLECtions to eLECT the eLECtors in the eLECtoral college.
teucer said,
November 19, 2012 @ 8:59 pm
Silver's state probabilities aren't independent, so the biased coins analogy doesn't actually illustrate how likely it is or isn't for him to get the results he did.