LL readers will not be surprised to learn that Daniel Casasanto and Kyle Jasmin disagree with my evaluation of their work on the "QWERTY effect". Yesterday afternoon, they added a comment to that effect on the original post. Since relatively few of the people who read that post are likely to see their comment, I'm reproducing it below. I'll respond at some later point.
The Robustness of the QWERTY Effect
It seems our study has caused quite an uproar on the Language Log (and the blogs that feed off of it).
We’re not concerned with Liberman’s subjective evaluation of the QWERTY effect’s size or of our study’s importance. We are concerned with his misrepresentation of the reliability of our findings.
Here’s the short version of our reply: The QWERTY effect is reliable. Replication is the best prevention against false positives. In this paper, we demonstrated the QWERTY effect *six times*: in 5 corpora (one of which we divided into 2 parts, a priori), in 3 languages, and in a large corpus of nonce words. We’ve replicated the effect again subsequently, showing it a seventh time, in a fourth language (see our full reply, URL below). In fact, we even find the effect replicated again (showing it for an eighth time) in the corpus by Dodds and al., when the analysis is appropriately controlled.
The analyses that Liberman presents on his blog post were performed, reported, and interpreted incorrectly. One of them is tantamount to flipping a coin 3 times and trying to make scientific inferences based on the results.
There’s a reason why scientific results go through peer review, and why analyses are not simply self-published on blogs. If there were a review process for blog posts, or if Liberman had gone through legitimate scientific channels (e.g., contacting the authors for clarification, submitting a critique to the journal), we might have avoided this misleading attack on this paper and its authors; instead we might have had a fruitful scientific discussion.
For a more detailed reply, see:
Daniel Casasanto and Kyle Jasmin