ICYMI: Globe summarizes Harvard report on Hauser
« previous post | next post »
Those who have been following the Marc Hauser case, on LLOG or elsewhere, may have missed this: Carolyn Y. Johnson, "Harvard report shines light on ex-researcher's misconduct", Boston Globe 5/30/2014:
When former Harvard pyschology professor Marc Hauser was found solely responsible in a series of six scientific misconduct cases in 2012, he distanced himself from the problems, portraying them as an unfortunate consequence of his heavy workload. He said he took responsibility, “whether or not I was directly involved.”
But a copy of an internal Harvard report released to the Globe under the Freedom of Information Act now paints a vivid picture of what actually happened in the Hauser lab and suggests it was not mere negligence that led to the problems.
The 85-page report details instances in which Hauser changed data so that it would show a desired effect. It shows that he more than once rebuffed or downplayed questions and concerns from people in his laboratory about how a result was obtained. The report also describes “a disturbing pattern of misrepresentation of results and shading of truth” and a “reckless disregard for basic scientific standards.”
The story was picked up by Jennifer Cousin-Frankel in Science magazine ("Harvard Misconduct Investigation of Psychologist Released", 5/30/2014), and Andrew Gelman ("Mmm, statistical significance . . . Evilicious!", 5/30/2014), but not otherwise widely noted.
Hauser has continued to get support from influential voices, including Nicholas Wade, who blurbed Hauser's latest book Evilicious:
What Steven Pinker has done for violence, Marc Hauser has achieved with evil – this book brings the light of science to illuminate the heart of darkness.
There's an interesting exchange between Norbert Hornstein and Andrew Gelman in the comments on Andrew's post "Evilicious: Why We Evolved a Taste for Being Bad", 6/16/2013. (The version below is excerpted to focus on the point of interest — see the whole thing at the cited link.)
Hornstein: Does A know the facts here? From what I could gather, Hauser was thrown out for supposedly cooking the books. Does A know that ALL of the disputed work has been replicated? Let me say that again: all of published work singled out for censure has been replicated. […] Maybe A thinks that replication doesn’t really matter, however. All that really matters is whether Hauser is “disgraced.” Whether he deserves to be is another matter entirely. Right? Just asking.
Gelman: See here for some details. Lots of falsification going on, not just “he may have been sloppy, he might have been able to do it better.” More like, he knowingly wrote untruths.
Hornstein: To repeat, all the disputed work replicated. There is nothing to retract. He was right. Not, he though he was, but from all we know right now, he was. For you this counts for nothing? Form over substance?
Gelman: You got it. I don’t want to have someone around who writes things he knows are false, who falsifies data, who publishes a graph of made-up numbers that purport to be real. […] To a statistician, the data are substance, not form. If you think that when a paper has fabricated data that “there is nothing to retract,” I think that’s just sad. I think honesty in reporting of data and measurement is central to science, and to scholarship more generally.
Hornstein: I resist being put into the nutty position of defending data fabrication. So, let’s stipulate that making up data is bad. But, I do think that ‘l’affaire Hauser’ has been vastly overblown. His alleged fakery did not disrupt the forward march I’d science. He was substantially correct. How do I know? Because to this date nobody has shown that his reported results do not stand. Cr all the huffing and puffing, for all the purported cooking of the books he was right. And not just once but all three times (I.e. the published work). So though I too would like people to report their findings truthfully and though like everyone else I think that honesty is the best policy, if one is interested in advancing science, the sin of fakery has not been, in my view, the main impediment to good scientific work.
Gelman: You write, “like everyone else I think that honesty is the best policy.” No, that’s not true. Not everyone thinks that honesty is the best policy. Marc Hauser did not think honesty was the best policy. He thought the best policy was to publish statements that he knew were false, to make up data and to falsely describe his data collection. Diederik Stapel did not think honesty was the best policy. Jonah Lehrer did not think honesty was the best policy. Dr. Anil Potti did not think honesty was the best policy. Etc etc etc. Lots and lots of people don’t think honesty is the best policy. As a noted psychology researcher once wrote, “we evolved a taste for being bad.”
Norbert's position seems to be that Hauser's original publications were "fake but accurate", in the memorably phrase used by a NYT headline writer to describe a different sort of situation …
My own view is that this case should be considered in the context of the whole spectrum of less-than-honest scientific practices. At one end there's subjective coding, cherry picking, data dredging, and model shopping; at the other end, there's outright fabrication. If strictures against all of these activities were enforced by dismissal or forced resignation, university science faculties would be sadly reduced.
I'm not trying to make a "slippery slope" argument, though I suspect that many data-fabricators start out with more innocent forms of scientific dishonesty. It's fine to treat making up data as a crime of a different order from confirmation bias in data coding, or the "file drawer effect" in deciding what to publish. But as a result, subjective coding and publication bias are probably bigger problems for science than fake data is.
Some of these problems will be somewhat ameliorated by the trend, pushed by government policies in the U.S. and the U.K., towards open data, as well as the spread of techniques for automating coding that used to require subjective judgment. Linguistics research is especially open to this change.
[The (redacted) pdf of the Harvard report is here.]
Neuro Polarbear said,
June 9, 2014 @ 6:32 am
Part of the problem is that Hauser has published a large amount of almost-too-good-to-be-true results, several of which have not been replicated. These results were not identified as possible fraud cases but then again, that doesnt mean they were cleared.
Carrington Dixon said,
June 9, 2014 @ 9:49 am
"If strictures against all of these activities were enforced by dismissal or forced resignation, university science faculties would be sadly reduced."
I should leave out "sadly". Perhaps, we should return to the days when university science faculties knew the difference between science and advocacy. I practiced each in its proper place.
Norbert said,
June 9, 2014 @ 11:05 am
I agree with the gist of your conclusion and I also think that among the sins of the trade actual cases of outright fabrication are the easiest to deal with, even if they are the easiest to morally condemn. As for Hauser, I do not believe that he actively fabricated data and I do believe that he could and should have been more careful (and maybe, more skeptical).
I want to make two additional points. First, that lots of experimentation is an art. Polanyi talked about this decades ago. The art is subject to abuse precisely because it is an art. So how do we know if someone is creating fakes rather than valuable originals? Well by whether others can get the reported results when they do the experiment. In other words, replication is the gold standard. If you think that this is so, then you also conclude that if an experiment replicates then there is a strong presumption that the original was conducted as it should have been. What look like cut corners and slapdash conclusions are vindicated by success. We conclude this for many of our scientific heroes, including Mendel, Millikan and Newton among others. I am not saying that Hauser is in this class, but I am saying that experimental hygiene is far less clear cut than we are inclined to admit.
Second, I take issue with your very last sentence. I actually think that linguistics, at least in my part of the world, is pretty robust wrt the data that it considers. I am sure that you've read the Sprouse work on this indicating that the informal collection methods that are standardly criticized in fact stand up surprisingly well when examined more stringently.
[(myl) In traditional "armchair" linguistics, the data is already open, because it's in the form of a list of morphological paradigms, or of the author's grammaticality judgments, or intuitions about meaning, or etc. This evidence is informal, and subject to criticism of various sorts, but it's clear what the evidence is (at least in accessible languages), and anyone is free to challenge it without coming up against a wall of secrecy.
In experimental and/or corpus-based linguistic research, we can aspire to a similar degree of openness, while adding somewhat better assurance against subjectivity and confirmation bias. In fact, this is already true for those forms of corpus-based research that depend on well-documented archives of texts in dead languages.]
Last point, re neuro polarbear: at least in the cases at issue, Hauser's results were replicated. If there are some that have not, what are they? I am always a little suspicious of claims that are critical but vague, especially when they report results that are clearly not in tune with what one might expect (too good to be true!). So, in this case in particular, vague accusations seem to me very suspicious.
bks said,
June 9, 2014 @ 11:13 am
Fake but accurate: So as long as your publications reflect the consensus of the reviewers, you cannot commit fraud? At least Hauser did not attempt to blame it on his students, as is the normal practice of tenured fraudsters. e.g.
http://www.nature.com/news/2009/091012/full/news.2009.998.html
–bks
bks said,
June 9, 2014 @ 11:21 am
pyschology? Mistake is in the original. –bks
ThomasH said,
June 9, 2014 @ 11:27 am
Of course if a researcher writes down and publishes data that is different from the results obtained it is fraud. The researcher lied about his results. Whether this is the most important problem with the way science is done is beside the point. Lying is lying. I had a professor who once said that there are sins and professional sins. Lying is a sin for anyone but a professional sin for academics.
Jerry Friedman said,
June 9, 2014 @ 12:03 pm
At one end there's subjective coding, cherry picking, data dredging, and model shopping; at the other end, there's outright fabrication. If strictures against all of these activities were enforced by dismissal or forced resignation, university science faculties would be sadly reduced.
But what if, starting today, people who had done those things didn't get academic jobs or tenure? There are plenty of applicants (I originally wrote "advocates" :-) for every position. Would faculties be reduced because vacancies wouldn't be filled? I rather doubt it.
Of course, it's easy for me to suggest something like that. I teach at a community college and don't have to evaluate anyone's research. But I wonder how much of the problem is that members of hiring committees don't have the time or inclination to evaluate people's research thoroughly and how much is that they don't think subjective coding, etc., are particularly sinful.
By the way, something else at the mild end of academic dishonesty is suppressing a result or even a line of inquiry because it would rebut the work of one's adviser, a member of one's tenure committee, etc. (Some people might think I said that out of painful experience, so I'll mention that my adviser encouraged me to make some objections in my thesis to a recent paper of his.)
Howard Oakley said,
June 9, 2014 @ 12:08 pm
How appropriate that this is about the psychology of evil.
Having dealt with a case of fabrication of data, and drafted a policy intended to address research governance issues such as data fabrication, I have always been amazed that 'management' really don't seem to care as much as (good) scientists do. I would dismiss instantly anyone found to have fabricated data, and make damn sure that they never worked in any field of science ever again. I think that management were keener to cover up or play down…
Howard.
J. W. Brewer said,
June 9, 2014 @ 12:13 pm
I would be curious as to how the various sarcasm-detection software packages that are presumably being developed (and/or existing such packages having their marketing materials refurbished) in response to the Secret Service's RFP would evaluate myl's line re "sadly reduced."
[(myl) In fact I mean the adverb literally in this case, but with (what I now see is) a non-obvious scope: If strictures against all of these activities were enforced by dismissal or forced resignation, university science faculties would be reduced; and the degree of this potential reduction should make us sad, because it means that a depressingly large percentage of scientists are guilty of such things.]
Richard W. Symonds said,
June 9, 2014 @ 1:17 pm
Based on the evidence, or lack of, an investigation into Harvard is
called for as a matter of urgency ("Harvard report shines light on ex
researcher’s misconduct", Boston Globe, May 30 2014)
http://www.bostonglobe.com/metro/2014/0 … story.html
The inconsistencies clearly evident in the Harvard Report raise very
serious questions regarding the motives of those involved in the
'Investigation' of Dr Marc Hauser.
For example:
1(a) "Hauser then wrote an e-mail suggesting the entire experiment
needed to
be recoded from scratch. “Well, at this point I give up. There have
been so many errors, I don’t know what to say. . . . I have never seen
so many errors, and this is really disappointing,” he wrote.
In defending himself during the investigation, Hauser quoted from that
e-mail, suggesting it was evidence that he was not trying to alter data.
The committee disagreed.
“These may not be the words of someone trying to alter data, but they
COULD certainly be the words of someone who had previously altered
data…"
1(b) "COULD" ??!!
2(a) 'Later that day, the person resigned from the lab. “It has been
increasingly clear for a long time now that my interests have been
diverging sharply from what the lab does, and it seems like an
increasingly inappropriate and uncomfortable place for me,” the person
wrote…'
2(b) Question : Who was that "person"?
Answer : "Much has been redacted from the report, including the
identities of those who did the painstaking investigation and those who
brought the problems to light".
I am reminded of two passages:
1. Matthew 7 v 5
2. "Many people presumably know that they have done something wrong
based on reactions by others, but don't admit to the wrongdoing or take
responsibility. Some of these people are excessively narcissistic, a
disorder that can bleed into the presidency…President George W. Bush
failed to admit to the public that he went to war with Iraq for reasons
other than the one concerning weapons of mass destruction…" ~ Marc
Hauser (Source : 'Moral Minds – How Nature Designed Our Universal Sense
Of Right And Wrong", Ecco 2006 – Page 155).
An investigation into Harvard – and beyond – should take place
immediately.
J. W. Brewer said,
June 9, 2014 @ 2:57 pm
Hauser had two separate problems — his relationship with Harvard, and his relationship with the federal government (which had funded some of the research in question). Hauser eventually opted to cut a deal with the government and agree to certain restrictions (which expire in 2015) on his involvement with federally-funded research. Had he opted to fight to the bitter end in hope of exoneration (i.e. by engaging in an internal appeals process at HHS after the initial finding of misconduct and then potentially going to court if that didn't improve things for him), it is probable that much more of the evidence (both that helpful to his position and harmful to his position) would have ended up in the public record. But Hauser elected not to pursue that strategy. Similarly, Hauser parted ways with Harvard without getting into a lawsuit which would also (if pursued to the bitter end rather than settled) have been likely to get a lot more information into the public domain. Perhaps that additional information would on net make Hauser look better, perhaps it would make him look worse.
I see that the "Evilicious" book is not put out by a traditional name-brand publisher. In these new-economy days where those institutions have lost a substantial amount of their old gatekeeper function that may not be as significant a red flag as it once might have been, but it would be interesting to know what if any attempts there were to shop the manuscript to traditional publishers (either "academic" ones or trade imprints with no qualms at all about putting out controversial books by scandal-tarred celebrities if there might be a dollar in it).
Although myl drew attention to that 2013 Andrew Gelman post because of the dialogue in the subsequent comments thread, the original post may also be of some interest to LL readers because of the observations Gelman makes about a Certain Prominent Personality in the linguistics world. (Your punchline about how since Chomskyanism is stereotyped as oblivious to empirical evidence not even a hypothetical dishonest Chomskyan would feel tempted to falsify data goes here.)
Richard W. Symonds said,
June 9, 2014 @ 3:31 pm
Viking-Penguin had agreed to publish “Evilicious”, but pulled out at the last moment – leaving Hauser no choice but to get it published by Amazon, Kindle et al.
I am growing in the conviction – with growing evidence – Viking-Penguin’s decision not to publish Hauser’s book was a very serious professional mistake – as well as an unforgivable breach of promise.
Viking-Penguin, with such a history of publishing courage and integrity, clearly caved in to pressure from un-named legal and political forces – just like Harvard.
“Evilicious” was the sequel to “Moral Minds” (Ecco HarperCollins 2006). In both books, Hauser has made a critical contribution within a ‘cutting-edge’ research field.
It is deeply saddening, and disturbing, to know there is unlikely to be an admission of wrong-doing either by Viking-Penguin, or by Harvard.
Jeffrey Goldberg said,
June 9, 2014 @ 4:40 pm
I have not been following this case. This is the first I've heard of it. (I've been out of academia for a long while.)
First it makes me very sad that my old friend Marc has been engaging in academic fraud. I really don't know how to respond. I, of course, condemn the fraud.
It also appears — at least from the posted article — that some people are attempting to use Marc's misconduct to discredit his whole line of research. If Norbert Hornstein is correct that all, or even most, of the fraudulent results have been replicated, then it really would be a mistake to discredit the line of research.
Although Marc's individual results must be thrown away, we can't dismiss his conclusions if there are more credible results supporting those. Marc's misconduct does not discredit the work done by others. That would be like rejecting genetics because Gregor Mendel may have cooked his pea plant data.
J. W. Brewer said,
June 9, 2014 @ 4:49 pm
Hauser appears not to be (or at least no longer to be?) black-listed from publishing articles on language-related subjects in peer-reviewed scholarly-sounding journals:
http://retractionwatch.com/2014/05/06/new-frontiers-marc-hauser-back-publishing-in-scientific-literature/#more-20290 (I am not familiar with the journal in question, but my own lack of familiarity is meaningless and nothing about the journal's website gives me an immediate vibe of bogosity.)
J. W. Brewer said,
June 9, 2014 @ 5:03 pm
OK, he's not publishing in a traditional old-economy type of peer-reviewed journal, but that doesn't (necessarily) mean it's a vanity-publishing scam versus just the wave of the open-access future: http://www.economist.com/blogs/babbage/2013/02/scientific-publishing
Richard W. Symonds said,
June 9, 2014 @ 5:05 pm
Hauser has been 'black-listed' from mainstream research – and mainstream publishing houses. His line of research has also been discredited. Of that there is no doubt for those with eyes to see.
One could be forgiven for thinking this was an orchestrated set-up – and then the 'attack-dogs' were let loose on him.
One could also be forgiven for asking who might have orchestrated the set-up.
But then we move into territory controlled by the 'dark side' – and we are forbidden to enter.
The truth, if it is ever exposed (which is extremely unlikely), will not be pretty.
Richard W. Symonds said,
June 9, 2014 @ 5:41 pm
Replications of Hauser's Research Work
http://hauserreplications.blogspot.co.uk
Ken said,
June 9, 2014 @ 9:12 pm
@bks: At least Hauser did not attempt to blame it on his students
The phrase "whether or not I was directly involved" is pretty close, though.
Jerry Friedman said,
June 9, 2014 @ 10:12 pm
bks: As far as I can see, nobody has said it wasn't fraud. What Norbert Hornstein argued was that the fraud wasn't harmful to the progress of science.
Richard W. Symonds said,
June 9, 2014 @ 11:29 pm
This might be an 'inconvenient truth' to some, but it should not be forgotten Harvard never fired Hauser. They had every intention of allowing him to teach and carry on his research – and Hauser had every intention to return – but something happened…
Can we honestly put our hand on our heart and say the punishment fits the crime in this case? You might. I can't. Hauser wasn't the only one at fault here. At least he admitted to his wrong-doing.
J.W. Brewer said,
June 10, 2014 @ 12:55 am
Just on the "open data" theme, the Globe story does provide (although not in a particularly intuitive place) a link to a pdf of the entire document they FOIA'd from the feds, i.e. the internal 2010 Harvard report, with redactions. The redactions seem to primarily and perhaps exclusively relate to the personal-privacy exemptions from FOIA disclosure. FOIA practice is a whole specialty I know very little about, so whether the agency was overaggressive in its redactions (and whether some of the redactions could be successfully challenged if the Globe or someone else were willing to push the issue some more) is not something I'm in a position to speculate about. But even with the redactions it's a pretty long document, and providing the link means that people are not at the mercy of the Globe's necessarily quite short summary and the related inherently subjective judgments about what the high points and quotable bits were.
It may be interesting to see whether the release of the report in redacted form will lead to a cascade of speculation and/or puzzle-solving in which those with relevant knowledge about who was working with Hauser on what project at what point in time are able to make and then publicize fill-in-the-blank educated guesses as to who is who behind the redacted names.
Note that to the extent Harvard itself as a matter of its protocols for such things (whether those protocols are optimal or suboptimal is another question) promised confidentiality to various witnesses etc., it may not be in a position to release an unredacted copy even if it wanted to (and likewise would not be in a position to tell the gov't – go ahead and release the whole thing – we don't care). Hauser presumably has an unredacted copy – whether he is bound (as a formal, legal matter) to confidentiality is another interesting question I don't know the answer to, and of course the answer might be unclear and he might be hesitant about conducting the experiment of putting it out there and seeing if he then got sued and if so what the outcome of the lawsuit was.
One wrinkle where I don't really know what to think because I don't know enough about the norms of the current academic community to have reliable intuitions is whether it's odd that not only the participants/witnesses in the underlying events (mostly grad students under Hauser's supervision, I would assume) have their names redacted but so do the members of the investigating committee. I assume the committee consisted of established academics with relevant backgrounds/specialties (maybe internal to Harvard, maybe external, maybe a mix) who are protected by tenure etc. I could see some benefit (at least assuming there continues to be general interest in the story) to a minimalist public statement by the committee members "outing" themselves, and basically saying "we don't necessarily want to comment generally or get into debates about the details, but under the circumstances we are willing to publicly confirm that this is our work, we stand by its conclusions, and we are willing for our professional peers in our discipline(s) to associate us with this report just as they associate us with our published scholarship and judge us accordingly." But that may well be a naive outsider's perspective, and there might well be sensible countervailing reasons why that would be imprudent.
However, I note by way of precedent that when the historian Michael Bellesiles engaged in career-destroying scholarly fraud, the report of the investigative committee convened by Emory to evaluate the situation (which is available in unredacted form on Emory's website) disclosed the names of the committee members, who were three well-known scholars in the relevant discipline, from three separate institutions outside Emory, at least one of whom had also been a university president. I tend to think that that transparency was helpful.
D.O. said,
June 10, 2014 @ 1:01 am
Maybe it's overly puristic on my part, but how fraudulent research can ever be replicated? Presumably, it means that the conclusion previously supported by fraudulent data is now supported by data obtained without fraud. But that's sort of beyond the point. It's not a replication of anything. It's just like saying that if police faked fingerprints of a murderer to get a conviction, but then real fingerprints of the same individual were found on the scene, the first set becomes somehow real.
Richard W. Symonds said,
June 10, 2014 @ 1:56 am
Here is Harvard's redacted full report:
http://cache.boston.com/news/pdfs/harvardreport.pdf
To say the redacted material (of name(s), reports, statutes) is "aggressive" is a serious understatement – which does raise an even more serious question as to who was responsible for the redactions.
The Globe's Carolyn Johnston has said "I can't speculate about the redacted names", which suggests she and the Globe – and Hauser & Harvard (& its students) – are legally bound not to make any mention of names…or else.
Presumably, this legal matter involves the legal team of Alan Dershowitz at the Harvard Law School, who were heavily involved at the start of the controversy?
Richard W. Symonds said,
June 10, 2014 @ 2:30 am
This might not be entirely irrelevant – "Decadence" by CEM Joad (1948 UK/1949 US), an English philosopher whose 'fall from grace' in 1948 was very rapid after the media-hyped train ticket 'scandal' – sacked by the BBC, but not by University of London's Birkbeck – the philosophy department which he established in 1930 until his death 23 years later, aged 61:
http://www.shuv.org/j_biblio.html
Joad, C.E.M. Decadence: A Philosophical Inquiry (New York: Philosophical Library, [1949]). The late C.E.M. Joad (1891-1953) was a renowned philosopher on the faculty of the U. London (1930-1953). He published many works, including studies in ethics, theology and religion. In this study Dr. Joad defines decadence as "the arrogance of man getting above himself and thinking he is lord of the universe" (pp. 14-15). He concludes that a decadent society is caused by the loss of belief in a higher spiritual reality: "the conviction that what is good and beautiful has its origin in and derives its authority from some other plane of reason… [which] invests our own world with a significance it had otherwise lacked." This conviction, Joad said, "forms part of the Christian tradition of Western Europe," but is threatened by factors which tend to preempt it such as "material prosperity and technical advance" (pp. 12-13). Characteristics of a decadent society include: "luxury, scepticism, weariness and superstition… as well as a preoccupation with the self and its experiences" (p. 117).
"A non-decadent community" Joad says, "is… one which is conscious of the spiritual order of the universe, more particularly as it manifests itself in values. Inhabitants of this order are God and the values in which God expresses and manifests Himself, namely, truth, goodness, and beauty…In decadent societies [this spiritual order] is lost sight of" (p. 281). Joad labels the materialistic cultural drift "insectification," and states, anticipating Thomas Cahill, and perhaps supporting Niebuhr's category: "Christ above culture,": "Granted the theistic hypothesis we may, I think… suppose that some human beings will be proof against the general process of 'insectification' and will retain the standards and values of civilized men, or will retain at least their memory much as the saints preserved the Christian faith in pagan lands and the monks some remnants of classical culture during the Dark Ages… By withdrawing, a man might…hope to keep alive some remnant of the culture of an earlier and more civilized time" (pp. 399; 401).
J.W. Brewer said,
June 10, 2014 @ 2:55 am
It is not clear to me how establishing that Harvard is decadent in a Joadian sense (not its face an implausible argument) would tend to vindicate or rehabilitate Hauer.
J.W. Brewer said,
June 10, 2014 @ 2:56 am
It is not clear to me how establishing that Harvard is decadent in a Joadian sense (not on its face an implausible argument) would tend to vindicate or rehabilitate Hauer.
Richard W. Symonds said,
June 10, 2014 @ 3:04 am
J.W., I was thinking less of Harvard and more of the Office of Special Plans under the Bush administration.
Richard W. Symonds said,
June 10, 2014 @ 3:14 am
Establishing "decadence" in the Joadian sense for certain high-level officials – Kissinger & Perle immediately come to mind – does not "vindicate or rehabilitate" Hauser – but it certainly makes his "misconduct" pale into insignificance.
Ken said,
June 10, 2014 @ 7:29 pm
In my post last night I was going to mention how much I dislike the "I didn't know what was happening, in this organization which I'm supposed to be running" excuse, but decided not to because there are surely worse excuses. And "yes, he's bad, but look how much worse those other guys are" is one of them.
Richard W. Symonds said,
June 10, 2014 @ 11:04 pm
But Ken, Hauser himself did not make one of your "worst excuses" an excuse, did he?
Anyway, I would seriously question your 'loaded' use of the word "excuse". One man's bad excuse is another man's good reason.
Hauser admitted mistakes were made – and took full responsibility for those mistakes. He did not admit to fraud.
Howard Oakley said,
June 10, 2014 @ 11:57 pm
With respect to the opinions that have been expressed, I think that a serious issue raised in the original posting is being missed. From the Globe summary (in reference to the 2005 paper):
"The committee painstakingly reconstructed the process of data analysis and determined that Hauser had changed values, causing the result to be statistically significant"
I was in a similar situation in the case which I had to investigate. In my case there were extenuating circumstances, as there often seem to be. The member of my staff (who was working on a far less scientific or worthy study) had undertaken a series of measurements on humans, which included 3 measurements of body mass. From those weights, a simple calculation determined one of the important outcome measures in the experiment. He had initially forgotten to perform one of those weighings on the first few subjects. Instead of admitting to the error and correcting it by discarding those results, and moving on with fresh subjects, he decided to fabricate the missing data. He did this quite well, but my suspicions were aroused as some of the results derived looked a bit too good to be true. I asked those who had been involved in the study, and they confirmed that the weighings had not been performed. I confronted the member of staff and he denied it. Thankfully as I progressed the formal investigation, he eventually (having filed a counter-claim that I was harassing him) admitted, and we agreed that he should resign with some remaining dignity. He left science, thankfully.
I am sorry, but I consider that (and, judging from the summary in the Globe, Hauser's case) to be clear falsification of data, and that such falsification of data is a research crime of the greatest severity. Issues over how others handle it are potentially of immense importance, but do not and cannot detract from the conclusion that someone working in scientific research in its broadest sense has attempted to lie through their data. In doing so, they have destroyed all credibility in everything that they have previously been involved with, and can have no credibility in the future. We cannot rigorously test all their previous work to determine which may have been similarly corrupted, nor can we devise a robust system to check their future work.
I am also afraid that whilst opening up original data to external scrutiny can sometimes allow others to question whether those data have been tampered with, ultimately openness in data is no solution to the problem: everyone in research has to be honest, and such honesty has to be enforced by a rigorous code of discipline within the community.
If taking effective action against such cases depleted academic staffs for a while, then so be it. It would not take long for the rest of the community to learn that, just like committing murder, you never never falsify data.
Howard.
Jonathan Badger said,
June 11, 2014 @ 12:46 am
This argument of "Fake but accurate" seems to be awfully similar to the defense that some have made for the late Cyril Burt's twin studies. Never mind that the raw data doesn't exist — the results seem plausible. I don't buy it. It's like how Morgan Roberton's 1898 novel "The Wreck of the Titan" somewhat predicted the actual 1912 sinking of the Titanic. Presumably both novelists and scientific frauds aim to some degree of plausibility.
Richard W. Symonds said,
June 11, 2014 @ 2:07 am
I would be persuaded by Howard Oakley's argument if – and only if – the name(s) of those involved in the "painstakingly reconstructed" (newspaper reporter) had not been painstakingly redacted.
Because of that painstaking redaction there is no way of knowing if the Committee (whose names are redacted) – basing their report on an individual (whose name is also redacted) – have been "painstaking" in their investigation.
In other words, another interpretation could be given to:
"The committee painstakingly reconstructed the process of data analysis and determined that Hauser had changed values"
The Committee could be guilty of the very thing for which they are accusing Hauser.
Brett said,
June 11, 2014 @ 10:22 am
As was pointed out in the comments on another recent post ("Comments," oddly enough), conspiracy theories tend not to explain why the purported conspirators are doing what they are doing. In this case, I see no plausible motive for Harvard to try to take Hauser down; in contrast, the motives for him to commit fraud are pretty obvious.
Richard W. Symonds said,
June 11, 2014 @ 10:33 am
Brett, some conspiracy theories turn out to be correct. Attempts to bring down Chomsky have been well-documented.
Anyone politically awake knows the reason why.
J. Goard said,
June 11, 2014 @ 10:46 am
(Your punchline about how since Chomskyanism is stereotyped as oblivious to empirical evidence not even a hypothetical dishonest Chomskyan would feel tempted to falsify data goes here.)
Unless it's convincing two generations of linguists who haven't read Verbal Behavior that you've honestly and accurately summarized its contents.
Richard W. Symonds said,
June 11, 2014 @ 10:54 am
JG, it would really help your case if you made yourself clear.
JW Mason said,
June 11, 2014 @ 3:17 pm
some conspiracy theories turn out to be correct.
This seems like a rather risky heuristic.
J. W. Brewer said,
June 11, 2014 @ 3:57 pm
JW Mason, there's a classic syllogism that goes something like: A) Galileo was scoffed at by the scientific establishment of his time and he turned out to be right; B) I am scoffed at by the scientific establishment of my own time; C) therefore . . .
J. Goard: since it was a joke rather than a full critique of Chomskyanism (don't worry — I have plenty of negative views about it!), it was limited to the "armchair" style, where the scholar sits in his office at MIT, introspects about whether given example sentences in English are or are not grammatical, and does no further analysis or collection of data outside his own head. The idea of deliberately misreporting your own subjective intuitions about grammaticality as a way of committing scholarly fraud seemed rather comical. Obviously, Chomskyan linguists like other linguists do write papers that make claims based on evidence from other languages, and if it's a language the reader (and perhaps more importantly, a particular other scholar that a journal may have asked to review the manuscript as part of pre-publication peer review) doesn't himself know, there is room for misleading (or worse) presentation of that evidence, and of course for some languages the entire discipline is necessarily relying on the competence and integrity of the handful of people who have actually done the fieldwork. I can remember discussions in a class I took circa 1986 about whether some particular Chomskyesque claim (one that I expect would no longer be coherent, because it assumed the deep/surface distinction which I have a vague sense did not survive the subsequent rise of the minimalist program) was inconsistent with the way ergativity worked in Dyirbal. Even at the time I was thinking "if this theory stands or falls on its ability to account for Dyirbal, we all better be hoping that Dixon's explanation of Dyirbal is empirically accurate."
Richard W. Symonds said,
June 11, 2014 @ 4:18 pm
Before disappearing up my linguistic a posteriori, may I just ask Brett @10:22am what "pretty obvious" motives would there be for Hauser to commit fraud. I can guess at certain motives – but they are not "pretty obvious" to me….just as I can guess at certain motives for Harvard.
Brett said,
June 11, 2014 @ 4:50 pm
@Richard W. Symonds: Producing strong research results is the best way for a scientist to advance their career. Strong results include those that are unexpected (but borne out by subsequent work), are difficult to obtain, and/or provoke a lot of interest. It's much easier to get these kinds of strong results by fakery than actual experimentation. I certainly could get a lot more attention for my research by fraudulently increasing the sensitivities by a couple orders of magnitude.
I'm afraid I cannot, on the other hand, guess a reasonable motive for Harvard (which is a large institutional bureaucracy and presumably not much swayed by interpersonal vendettas).
Richard W. Symonds said,
June 11, 2014 @ 5:02 pm
@Brett
"I’m afraid I cannot, on the other hand, guess a reasonable motive for Harvard"
So the fact Alan Dershowitz of Harvard Law School was involved in giving legal advice in the early stages doesn't raise an eyebrow?
Richard W. Symonds said,
June 11, 2014 @ 5:15 pm
This was written in October 2010 – which raises my eyebrow (if not anyone else's):
http://blogs.discovermagazine.com/neuroskeptic/2010/10/30/marc-hauser-plot-thickens-solidifies-cracks/#.U5jT5xa19kw
Breffni said,
June 11, 2014 @ 5:27 pm
JWB: That fallacy was neatly skewered by Carl Sagan: "They laughed at Columbus, they laughed at Fulton, they laughed at the Wright brothers. But they also laughed at Bozo the Clown."
While I'm here, I'd like to endorse D.O.'s point above: it doesn't make sense to say an untrustworthy result has been 'replicated', since 'replication' normally presupposes that the original result was itself valid evidence of the effect in question.
J. W. Brewer said,
June 11, 2014 @ 6:11 pm
I would note that while the release of the report in redacted form is no doubt frustrating to those interested in the story, because it presents substantial information not previously in the public record but also presents that information in an obviously incomplete form (with the redactions perhaps making the omitted information seem particularly tantalizing), this is simply a result of how FOIA works. If Mr. Symonds or others find the mere fact of the redactions puzzling or suspicious, that is a sign that they lack relevant prior experience with the FOIA process (which is not to be taken as damning — it's a comparatively obscure corner of modern American bureaucratic life that there's no reason for most non-specialists to know about unless they've had particular prior occasion to deal with it). The key point here is that the media did not get it from Harvard, they got it from the federal government, which in turn had gotten it from Harvard quite likely under conditions where it (the government) was obligated to maintain it in confidence except to the extent FOIA or a court order dictated otherwise. (Exactly what parts of the report the law permits and/or requires to be redacted under such circumstances may involve judgment calls by the government personnel doing the redactions, and if the Globe or some other media outlet with access to lawyers and willingness to pay them followed up and pushed a bit harder, it is possible in principle that some but likely not all of the redactions might be revisited.) The follow-up questions for those unhappy about the redactions would be:
1) Is Harvard free to put out an unredacted copy if it wants to (both as a question of external law and of its own internal official policies)?* If it is but declines to do so upon reasonable request, is that itself suspicious? (You'd need to know a lot about Harvard's past practices with regard to the confidentiality of similar documents, under what if any circumstances previous exceptions have been made, etc. to answer that last question.)
2) Is Hauser himself free to put out an unredacted copy if he wants to? If so, what inferences may be drawn from his failure to do so?
3) Does someone who is close to the situation and generally supportive of Hauser have an unredacted copy (possibly previously obtained through informal or irregular means) that could be leaked either to an interested media source or via the internet more generally with Hauser himself having a plausible basis to deny responsibility for the leak and the leaker himself having a reasonable chance of (if desired) avoiding exposure of his identity? If so, what inferences may be drawn from the failure of such a leak (thus far) to occur?
*I noted earlier in the thread that Harvard's approach to this report seems to be very different from Emory's approach to their report investigating Bellesiles, which is posted in unredacted form on Emory's website. I do not know enough about how American universities typically handle allegations of scholarly misconduct by faculty to be able even to guess which of these instances represents the rule and which the exception, or if there are other distinguishing factors that make each approach normal/typical in the specific context of the issues being investigated.
[(myl) Note that some of the people involved were students, and therefore Harvard is prohibited from disclosing their connection to the situation by FERPA.]
Brett said,
June 11, 2014 @ 7:58 pm
@Richard W. Symonds: I have no idea how Dershowitz being involved in the case is supposed to suggest that Harvard has a motive to screw Hauser over.
On the subject of "replication" of falsified results: Any such "replication" is also going to be a little suspect. When you know what the result is "supposed" to be, it's easy for the results to be a bit tilted, even if it's not done intentionally. This is especially true when the original result was produced by somebody quite famous. I am certainly not a big shot from a famous university, but I have a reputation for expertise within in my own small corner of academia. I've seen my own work completely "replicated" by more junior people even when it turned out to contain a mistake.
There is also an interesting manifestation of the "file drawer effect," which makes it difficult to publish negative results. If you want to extend someone else's research, it's frequently necessary to duplicate what has already been done. If you successfully replicate the experiments, you can publish that, along with further work elaborating on it. If you are unable to replicate the original result, you can't go any further, and it's not likely to be published.
Ken said,
June 11, 2014 @ 11:43 pm
@Brett: I have no idea how Dershowitz being involved in the case is supposed to suggest that Harvard has a motive to screw Hauser over.
It's guilt by association, yet another fallacious argument. It almost looks as if he's working from a list.
Howard Oakley said,
June 12, 2014 @ 12:01 am
If and when there is evidence that the data have been altered, there are three main reasons that are often advanced.
The first, which seems to be that of Hauser's evidence quoted in the Harvard report here, is of error arising from confusion. I have seen this in several claimed accounts, and each time find it extraordinary that anyone working so hard to win valuable data by way of profession should apparently be so careless with it the moment that they have it on a computer. In each case, if they really were that callous and disorganisation, then they lack the discipline to be involved in such work, and should leave it to a better organised student.
The second is the personal promotion motive suggested by Brett above, that they wish to publish important work which will build their reputation, enhance their funding, and so further their career. Although that may be the case for some, even in these dark days of the commercialisation of academia, I think that it is uncommon.
The third, which no-one seems to have mentioned above, is that most researchers are strongly driven by the hypotheses which they form and hold dear. Having designed and conducted experiments to test those, there is an irresistible desire to make the data fit the hypothesis. I have seen this so often, in the most junior and inexperienced, right up to my own role models. Outliers get suppressed, or excuses are found for disregarding them; observations are interrogated time after time to try to make them fit expectation better; spreadsheets are massaged or worse. Sadly this is unashamed data fabrication, and regardless of the high motives, remains the same fundamental crime as if for personal promotion.
Finally, the issue of duplication. However rigorous such duplication might be, it is completely irrelevant. If a scientist fabricates or falsifies data, those remain lies. The next time that they do it, duplication might not happen.
Howard.
Howard Oakley said,
June 12, 2014 @ 12:07 am
(Sorry, I had to rush off – a couple of quick corrections to my comment above.
"that really were that callous and disorganisation" should of course read "that really were that callous and disorganised".
In the final para, please substitute "replication" for "duplication".
Howard.)
Richard W. Symonds said,
June 12, 2014 @ 2:23 am
Howard O. says: "most researchers are strongly driven by the hypotheses which they form and hold dear. Having designed and conducted experiments to test those, there is an irresistible desire to make the data fit the hypothesis….Sadly this is unashamed data fabrication, and…remains the same fundamental crime as if for personal promotion"
I would disagree with that statement above in the strongest possible terms. John 8 v 7 immediately comes to mind : "Let he who is without sin cast the first stone".
A theorist has a theory – and builds up evidence to support that theory, (eg "make the data fit the hypothesis") – which is then tested. That is NOT "data fabrication" and NOT a "fundamental crime". If it is, then we are all guilty.
It only becomes a "fabrication" and "crime" if the theorist MAKES UP the data without evidence.
Hauser, as I see it, was building up evidence to support theories – he was NOT making it up.
Evidence to show he was not making it up would be replication, and the quality of his students as they progress in their careers.
Hauser appears to come out very well in these two respects.
Richard W. Symonds said,
June 12, 2014 @ 3:14 am
Replications of Hauser's research:
http://hauserreplications.blogspot.co.uk
Richard W. Symonds said,
June 12, 2014 @ 3:17 am
Some of Hauser's best students in the field hold jobs at Yale, Princeton. Dartmouth etc
Howard Oakley said,
June 12, 2014 @ 3:55 am
Richard,
Please read the context in which I wrote what you then quote out of that context.
There are many many scientists who follow the scientific method and publish sound conclusions derived from clean data. I am afraid that – whether replicated or not, irrespective of the achievements of their students, and all other irrelevancies – there are some who do not follow the path of honesty in their dealings with data. I gave, from my experience, three different routes that are claimed to accomplish such apparently flawed or fabricated data.
The Harvard report, redacted or not, makes very clear and detailed allegations about the fabrication of data by Hauser, for whatever reason. It details Hauser's own comments to the committee in trying to account for the discrepancies which it identifies. In that report, Hauser's account is judged not to provide either defence against the charge of data fabrication, nor any sound reason as to how that might have occurred other than wilfully. No eyebrows are raised, nor innuendos concocted.
I presume if the report does Hauser such an injustice, that he has published a detailed rebuttal, in which he establishes that the data were not fabricated? I would greatly appreciate a link to that, please, which is much more relevant and meaningful than whether others might have replicated his results, which says nothing of the veracity of those results.
Howard.
Richard W. Symonds said,
June 12, 2014 @ 4:31 am
Then, as I see it, it boils down to what is said at the conclusion of this 2010 article:
http://blogs.discovermagazine.com/neuroskeptic/2010/10/30/marc-hauser-plot-thickens-solidifies-cracks/#.U5lyn381gv4
Richard W. Symonds said,
June 12, 2014 @ 1:05 pm
"Altmann correctly says that either Harvard are wrong, or Hauser falsified data….we (including Altmann) seem to be left with a question of whether to trust Harvard University and their internal investigation"..or Hauser.
J. Goard said,
June 14, 2014 @ 3:22 am
@J. W. Brewer:
My remark was rather orthogonal to your points. I don't object to armchair grammaticality judgments as evidence, provided that peer review includes several native speakers of the language in question confronting the examples with some measure of skepticism.
My own half-joke was that Chomsky's famous review is so grossly unfair to Skinner's book that its status as an an academically published book review is something akin to a research paper with falsified data. Per the humorous tone several others have taken, I don't exactly believe this, although I do think it's a terrible review, and that far too many people sing its praises who haven't even read the damn book.
Richard W. Symonds said,
June 14, 2014 @ 3:46 am
@J. Goard
May I suggest you do not write in "half-jokes" – some people might not understand the (half) joke.
Skinner made the mistake of treating human beings as though they were equivalent to inanimate objects, such as stones.
Chomsky exposed that nonsense – and rightly so.
Richard W. Symonds said,
June 14, 2014 @ 5:33 am
Put more simply:
CHOMSKY – WE ARE NOT A 'BLANK SLATE' (TABULA RASA). FREE WILL IS NOT AN ILLUSION
BF SKINNER – WE ARE A 'BLANK SLATE' (TABULA RASA). FREE WILL IS AN ILLUSION
Rubrick said,
June 16, 2014 @ 4:50 pm
I'm late to the table and more than enough of substance has been said, so I'll just add that "What Steven Pinker has done for violence, Marc Hauser has achieved with evil" has got to be one of the best out-of-context sentences I've run across.
Richard W. Symonds said,
June 16, 2014 @ 10:00 pm
Kubrick, I would call that quote by Nicholas Wade probably the best summing up of the book. Wade says it as praise (which is on the back cover of "Evilicious"), but I would say it as criticism. Hauser attempts to explain away "Evil", and fails – just as Pinker attempts to explain away "Violence" in his book, and fails.
I am defending Hauser in many ways – but not his book "Evilicious". It's a very disappointing sequel to his "Moral Minds".
Richard W. Symonds said,
June 17, 2014 @ 12:03 am
To make real sense of Hauser's "Evilicious" and "Moral Minds" (& Pinker's), they have to be seen in the context of US political and military events of the time.