Ethnography of academia
« previous post | next post »
ICYMI — Alexander C. Kafka, "‘Sokal Squared’: Is Huge Publishing Hoax ‘Hilarious and Delightful’ or an Ugly Example of Dishonesty and Bad Faith?", The Chronicle of Higher Education 10/3/2018:
Reactions to an elaborate academic-journal hoax, dubbed "Sokal Squared" by one observer, came fast and furious on Wednesday. Some scholars applauded the hoax for unmasking what they called academe’s leftist, victim-obsessed ideological slant and low publishing standards. Others said it had proved nothing beyond the bad faith and dishonesty of its authors.
Three scholars — Helen Pluckrose, a self-described "exile from the humanities" who studies medieval religious writings about women; James A. Lindsay, an author and mathematician; and Peter Boghossian, an assistant professor of philosophy at Portland State University — spent 10 months writing 20 hoax papers that illustrate and parody what they call "grievance studies," and submitted them to "the best journals in the relevant fields." Of the 20, seven papers were accepted, four were published online, and three were in process when the authors "had to take the project public prematurely and thus stop the study, before it could be properly concluded." A skeptical Wall Street Journal editorial writer, Jillian Kay Melchior, began raising questions about some of the papers over the summer.
Jennifer Schuessler, "Hoaxers Slip Breasturants and Dog-Park Sex Into Journals", NYT 10/4/2018:
One paper, published in a journal called Sex Roles, said that the author had conducted a two-year study involving “thematic analysis of table dialogue” to uncover the mystery of why heterosexual men like to eat at Hooters.
Another, from a journal of feminist geography, parsed “human reactions to rape culture and queer performativity” at dog parks in Portland, Ore., while a third paper, published in a journal of feminist social work and titled “Our Struggle Is My Struggle,” simply scattered some up-to-date jargon into passages lifted from Hitler’s “Mein Kampf.”
Such offerings may or may not have raised eyebrows among the journals’ limited readerships. But this week, they unleashed a cascade of mockery — along with a torrent of debate about ethics of hoaxes, the state of peer review and the excesses of academia — when they were revealed to be part of an elaborate prank aimed squarely at what the authors labeled “grievance studies.”
The hoaxers explain themselves — James Lindsey, Peter Boghossian, and Helen Pluckrose, "Academic Grievance Studies and the Corruption of Scholarship", Areo 10/2/2018:
Our approach is best understood as a kind of reflexive ethnography—that is, we conducted a study of a peculiar academic culture by immersing ourselves within it, reflecting its output and modifying our understanding until we became “outsiders within” it.
Read the whole thing for details of what they did, and why, and what they think it means.
Some varied reactions:
Is there any idea so outlandish that it won't be published in a Critical/PoMo/Identity/"Theory" journal? Helen Plucrose et al. submitted a dozen hoax papers to find out. https://t.co/TTDLuIQN9p via @areomagazine
— Steven Pinker (@sapinker) October 3, 2018
Substantive merits of these fields aside, what strikes me about stunts like this is their fundamental meanness. No attempt to intellectually engage with ideas you disagree with; just trolling for lulz. I wish we could be better.https://t.co/Yn2t7sWisu
— Sean Carroll (@seanmcarroll) October 4, 2018
Wally w said,
November 8, 2018 @ 10:28 am
It seems to me that for these papers to be all rejected the journals would have to have policies of rejecting ideas far out of the mainstream. It’s easy enough to find ideas that were mocked as absurd until they were accepted as fact
Suzanne Valkemirer said,
November 8, 2018 @ 11:22 am
I agree with Wally w said. The experiment should have involved (alleged) facts, not ideas.
cameron said,
November 8, 2018 @ 12:09 pm
See this article for some interesting commentary on the case: https://slate.com/technology/2018/10/grievance-studies-hoax-not-academic-scandal.html
Many of the papers that were actually accepted were ostensibly based on empirical results derived from data analysis of bogus studies. This prank tells us a lot more about how impressed journal editors are by a veneer or statistics than anything about ideology.
jean-Sébastien Girard said,
November 8, 2018 @ 4:35 pm
I'm about 80% sure that the post by yoandri dominguez is some form of spam.
Michele Sharik Pituley said,
November 8, 2018 @ 11:38 pm
I agree with Sean Carrol’s tweet. If you disagree with something, make a substantive argument instead of engaging in a prank like this. This is supposed to be academia, an exchange of ideas, not kindergarten.
In other, more petty, news: color me shocked that they’re white. Sigh.
R. Fenwick said,
November 9, 2018 @ 12:25 am
The Slate article shared by cameron hits the nail firmly on the head, and for me this one line is the most important:
These examples haven’t hoodwinked anyone with sophistry or satire but with a simple fabrication of results.
To take the topic of this blog as an example, I'm fairly sure that it'd only take a modicum of effort for a malicious linguist to get a "groundbreaking" paper out of fabricating the existence of an entire language. The acceptance of a non-zero number of these papers might be cause for a certain amount of self-examination by academic publishers, it's true, but the fact is that two-thirds of the papers they concocted were never accepted for publication in the first place, and all these "headline grabbers" were based upon claims of actual fieldwork and data, which in reality never existed. Jacob Levy at McGill nails the target on Twitter: "I am so utterly unimpressed by the fact that an enterprise that relies on a widespread presumption of not-fraud can be fooled *some of the time* by three people with Ph.D.s who spend 10 months deliberately trying to defraud it." (Many of the comments in the ensuing thread also make useful points.)
Ursa Major said,
November 9, 2018 @ 6:11 am
I agree with some of the other commenters. Sokal's paper was filled with nonsense but didn't claim the author did anything other than think, it exposed overreach of the editors and reviewers into a field they knew nothing about. If a paper claims to have experimental results then editors and reviewers take it on trust that the work has been done. If there is nothing wrong with the claimed methodology and the claimed results support the conclusions then the paper has passed some of the biggest hurdles to publication.
Mark Paris said,
November 9, 2018 @ 10:28 am
When global warming (or climate change, if you prefer) deniers make claims about conspiracies and not being able to trust scientific studies, I sometimes reply that most reputable scientists assume that other scientists are working and reporting what they find in good faith. And I believe it's true. Most scientists do what they do in good faith. That is, perhaps, one reason that scientists can be fooled, at least for a while, by illusionists and charlatans; they don't expect people to be dishonest.
J.W. Brewer said,
November 9, 2018 @ 11:18 am
I was pleased to learn from this headline that the scientific community's current methods for evaluating bold new hypotheses include not only peer review and attempts to replicate results, but the eruption of shouting matches. https://www.salon.com/2018/11/08/scientific-shouting-match-erupts-over-whether-%CA%BBoumuamua-was-an-alien-spacecraft/
Ursa Major said,
November 9, 2018 @ 11:56 am
shouting match = vigourous peer review ;)
tangent said,
November 10, 2018 @ 12:50 am
I have to say the dog park study doesn't seem unreasonable to publish, if you actually had that data. Not a major result or anything, but it might be interesting to compare the real effect versus another place or time. No reason not to publish it.
Lying about doing work that would be publishable is different that sending text that means nothing.
szopen said,
November 10, 2018 @ 4:45 am
"If you disagree with something, make a substantive argument instead of engaging in a prank like this."
In the context of the recent affair, where a respected scientist couldn't publish his article, his coauthors one by one withdrew their names, and when finally paper was accepted, some activists used their influences to blackhole the already accepted and published paper… this sound really ironic (https://quillette.com/2018/09/07/academic-activists-send-a-published-paper-down-the-memory-hole/)
And I do not even mention the reactions to Strumia paper.
TIC said,
November 10, 2018 @ 3:17 pm
On the topic of identifying hoaxes (in particular, in the form of spam comments) I'm sad that I arrived too late to see the suspect, and jettisoned, yd comment… I'd have liked to have seen whether this comment looked any more or less like the product of a gibberish-generating spambot than any other…
J.W. Brewer said,
November 11, 2018 @ 3:55 pm
All hoaxes and practical jokes etc. more or less by definition involve an element of deceit. How do we decide which are hilarious and which are in bad faith and/or bad taste? Is it as simple as whether we regard the sucker who fell for it as a political ally versus a political adversary? Or is there something less reductionist than that?
Barbara Phillips Long said,
November 12, 2018 @ 10:21 pm
My introduction to this set of authors and their papers came through the Slate article. I notice most of the quoted matter in this post comes from publications that tend to serve academia and the well-educated reader.
My first concern is one I have had for as long as I have lived on the fringes of academia by virtue of living or working near college towns. I am unenthusiastic about the pressure to formally publish frequently, particularly if it institutionalizes marginal scholarship through slapdash peer review. The recent discussions about replicating psychological research contributes to my concerns. That said, because I am on the outside of academia, I have to rely on academics to parse these issues for me, and seeing comments such as those (in tangent's comment) that can be misinterpreted as defending a parody paper's serious consideration because there was data presented tells me that peer review is inadequate for fact-checking. Maybe it is time to slow down the publication timetable in order to allow for a better review process, perhaps by hiring unemployed doctorate holders as editors and fact-checkers.
My next concern is about the economics of academic journals. Would journals be able to afford more editors and fact-checkers if academic papers could be formatted to serve both the general reader and the academic specialist, so that more members of the public read academic research directly instead of encountering it filtered through various mass media? Academics should not be put in the position of chasing clicks, but their work should not be inaccessible, either.
My overarching concern is that advances in knowledge continue, that people read about them, that the new facts and perspectives are tested, corrected, and refined over time, and are accepted by the general public. To spread knowledge, academia needs to be more concerned with limiting jargon and improving writing.
I worry about anti-intellectualism in the U.S. Sometimes I fall prey to it. It seems to me that this parody stunt will just reinforce anti-intellectual trends that oppose fact-based research and cultural inquiry. One of my more vivid memories of my journalism career, which did not start until I was in my forties, was negotiating a profile I wanted to do about a local businessman who had an interesting background but was profoundly distrustful of newspapers, television, and scientists. One of our discussions veered off into a discussion triggered, I think, by Senator William Proxmire's Golden Fleece awards. The businessman and a friend of his were bellowing with laughter about research into the amount of methane gas produced by dairy cattle and making fart jokes. I was embarrassed by the jokes but somewhat sympathetic about the research topic because it seemed to me to be a minor detail of bovine digestion. I knew from my father's work as a veterinarian that there was still a lot to learn about cows, so I defended the research instead of joining the laughter. I pointed out someday we might need to know about how methane affected the air around us. I did not know until years later that measuring methane would be a component in climate change research. I had simply been brought up to believe that basic research was an essential good for the nation.
I look at the parody papers as yet another nail in the coffin of scientific credibility, whether or not that is what the authors intended. It seems to me that the more urgent question is how do academia, the U.S. education system, and ordinary people come together to appreciate and understand scientific inquiry? Cleaning house at academic journals is only part of that task.